Parallel Multi-splitting Proximal Method for Star Networks

Size: px
Start display at page:

Download "Parallel Multi-splitting Proximal Method for Star Networks"

Transcription

1 Parallel Mult-splttg Proxmal Method for Star Networks Erm We Departmet of Electrcal Egeerg ad Computer Scece Northwester Uversty Evasto, IL 600 Abstract We develop a parallel algorthm based o proxmal method to solve the problem of mmzg summato of covex (ot ecessarly smooth) fuctos over a star etwork. We show that ths method coverges to a optmal soluto for ay choce of costat stepsze for covex objectve fuctos. Uder further assumpto of Lpschtz-gradet ad strog covexty of objectve fuctos, the method coverges learly. I. INTRODUCTION We cosder the followg class of optmzato problem m x = f (x), whch has gaed much research atteto recetly. It captures may mportat applcatos such as dstrbuted cotrol for a team of autoomous robots/uavs pursug/amg at a commo target, sesor etworks costructg a estmato for the etre surroudg, commucato systems maxmzg system throughput, ad mache learg applcatos [5], [0], [6], [5], [9], [], [7]. Most of the exstg lterature for solvg ths problem ether does ot explore parallel potetal [6], [5], [9], [] or requres a careful selecto of stepsze to guaratee covergece [7], []. The requremet of stepsze tug ca be computatoally expesve, ad udermes the robustess of the etre system to provde a optmal soluto. The oly le of dstrbuted algorthms that does ot suffer from the drawbacks of stepsze selecto s Alteratg Drecto Method of Multpler (ADMM) based algorthms [3], [5], [0], [], [], [0], [3], [5], [], [], [8], [4], whch has gaed much popularty due to great umercal performace. A closer look of the stadard ADMM reveals that t s a two-way splttg proxmal algorthm [6], where a two-way splttg of the dual fucto s formed ad the proxmal method s appled teratvely to both parts. However, as observed recet work [4], whle the stadard two-way splttg ADMM (correspodg to a two-aget settg a mult-aget setup) ca coverge for ay stepsze choce, a three-way splttg of the dual fucto may result a algorthm that dverges. Hece, order to use ADMM a dstrbuted settg wth more tha two agets, complex reformulato of the problem ad troducto of auxlary (prmal ad dual) varables are requred [], [3], [0]. Despte the two promsg features that proxmal-based methods do ot requre stepsze selecto ad that mult-splttg arses aturally from mult-aget setup, the questo of whether we ca desg a coverget algorthm based o more tha two-way splttg proxmal method remas ope. I ths paper, we combe deas from proxmal method ad projecto to develop a mult-splttg proxmal algorthm that works wth o-smooth covex objectve fucto, takes advatage of parallel processg power ad guaratees asymptotc covergece for ay postve stepsze. We also aalyze ts rate of covergece uder stroger assumptos of Lpschtz gradet ad strog covexty ad show that the algorthm coverges learly. Our paper s related to the large lterature o dstrbuted/parallel computato, buldg upo semal works [] ad [9]. I partcular, the dstrbuted gradet descet method [] ad EXTRA [7] method. The dstrbuted gradet method ca be appled to o-smooth objectve fucto, however a costat stepsze would oly guaratee covergece to a error eghborhood of the optmal soluto. The recetly proposed dstrbuted frst-order method, EXTRA, uses a costat stepsze ad coverges to a optmal pot. However, the algorthm does requre careful selecto of stepszes to guaratee covergece ad smoothess of the objectve fuctos, whch lmts ts applcablty to mportat problems wth o-smooth regularzato term, such as the LASSO. The most closely related lterature for our algorthm s [8] from 983, whch was later geeralzed [7]. These authors combe mult-splttg proxmal method ad projecto to form a ew algorthm. Spgar s algorthm s a specal case of our algorthm wth a ut stepsze. We also ote that these papers do ot have rate of covergece aalyss (uder Lpschtz gradet ad strog covexty assumptos). The proposed algorthm shares the same rate of covergece as some exstg algorthms, such as EXTRA ad ADMM, the ma advatage s ts robustess agast stepsze selecto ad smple mplemetato dstrbuted settg. Whle we focus o the star etwork ths work, ths serves as a buldg bloco develop dstrbuted methods for geeral etwork topologes. The rest of the paper, we wll frst preset the algorthm alog wth some prelmary smulato results ad the the covergece ad rate of covergece aalyss. II. ALGORITHM We preset the proposed algorthm ths secto. Frst, we ote that the orgal problem ca be equvaletly expressed Due to space lmts, some proofs are omtted here. Iterested readers are referred to the author s webste for a mauscrpt cludg all the proofs.

2 Aggregator f (x ) f (x ) f 3 (x 3 ) f (x ) Fg.. Parallel archtecture to mplemet the proposed algorthm Mult Splttg (c=0.) DGD (α = 0.05) EXTRA (α = 0.05) Fg.. Prelmary umercal result. Y-axs: relatve error f(x t ) f f(x 0 ) f, x-axs: terato cout. as m x f (x ), = s.t. x =... = x. () We adopt a geeral setup where each fucto f : R m R s covex but ot ecessarly dfferetable. We am at developg algorthm to solve ths reformulated problem uder the followg stadard assumpto. Assumpto : Problem () has a o-empty optmal soluto set, deoted by X. Ths codto does ot requre uqueess of the optmal soluto. The parallel algorthm s mplemeted o + maches, coected a star graph as show Fgure. We call the oe the ceter the aggregator, the rest of them workers labeled {,..., }. Each worker has formato about oe fucto f ad specalzes computg the proxmal operator related to f. Collectvely, the workers ad aggregator are solvg problem (). Our algorthm s a teratve method, where the updates are related to decso varable ad frst order formato. At each terato, the workers parallel perform a proxmal pot updates for ther respectve f usg the curret state formato receved from the aggregator (related to decso varable ad correspodg frst order formato) ad seds the updated formato to the aggregator. The aggregator the averages the formato accordg to a specfc rule ad seds the averaged formato as the ew state baco each worker. I partcular, we use The umber of maches ca be reduced va m-batchg ad/or requrg the aggregator to also process formato about oe of the fucto f. We chose to preset the settg wth maxmum parallelsm. the followg set of otato to descrbe our algorthm. I our algorthm, we use superscrpt to dcate the terato cout ad subscrpt to dcator the worker that s assocated wth the varable. Postve parameter c s the stepsze ad s a costat throughout the algorthm. Our algorthm s preseted Algorthm. Algorthm Parallel Mult-splttg Proxmal Method Italzato: The aggregator starts from arbtrary x 0 ad ṽ 0 R m for,...,, compute x 0 = x0, for all =,... ad v 0 = ṽ0 = ṽ0. The aggregator seds formato x 0 + cv0 ( Rm ) to each worker. Iterato: for k = 0,,... Worker =,... computes parallel y k+ argm f (p) + p x k cv k, (a) p c w k+ = c (xk + cv k y k+ ), (b) ad reports y k+ ad w k+ (each R m ) baco the aggregator. After recevg y k+ ad w k+ formato from all workers, the aggregator geerates x k+ = = yk+, v k+ = w k+ = wk+, for =,..., ad the seds formato x k+ cv k+ ( R m ) to each worker. + The {y k} k sequece ca be vewed as local estmates of x. At each tme stat k, x k s the same for all ad equals to the average of all local estmates. The {w k } sequece as show later Lemma 3., represets a local subgradet assocated wth x k+ of fucto f. The varable v k captures the dfferece betwee local subgradet ad the average of all subgradets. Ths algorthm s well suted for problems where step (a), mmzato related to oe compoet of the objectve fucto, ca be mplemeted a effcet way. Examples clude SVM, quadratc objectve fuctos, Lasso (Least Absolute Shrkage ad Selecto Operator) (see [3], [4] for more examples). Whe aalyzg covergece speed for ths algorthm, we focus o the terato cout of k, ad ot coutg the tme eeded to solve step (a). We have performed some tal umercal studes to compare our method agast dstrbuted gradet descet (DGD) [] ad EXTRA [7] wth = 4, m = ad quadratc objectve fuctos. We plot the relatve error objectve fucto Fgure. We used stepsze of 0. for the proposed method ad 0.05 for DGD ad EXTRA,as they both dverge for the stepsze choce of 0. ad eeded smaller stepsze. III. CONVERGENCE ANALYSIS I ths secto, we aalyze the covergece ad speed of covergece of the proposed algorthm. For cocse represetato, we troduce the followg otato. Vector

3 x k = [x k ] R m s a log vector formed by stackg x k,.e., x k = x k. x k. Smlarly, we form vectors y k = [y k ], v k = [v k] ad w k = [w k], all R m. Uless otherwse specfed, vectors such as x wth sub-dces le R m ad those wthout sub-dces, such as x k are R m. We deote by F : R m R, ad F : R m R m, the x f (x ) x mappgs F. =., F. = x f (x ) x v., v f (x ),where the otato f (x) deotes v the subdfferetal set,.e., the set cossts of all subgradet of f at pot x. We use x y to deote the er product betwee two vectors x ad y. We ext show that our algorthm has two compoets: proxmal method ad projecto, whch serves as the bass for covergece aalyss. A. Proxmal method We start by aalyzg the sequeces y k ad w k. Step (a) ca be equvaletly expressed as y k+ prox cf (x k + cv k ) usg defto of proxmal operator. We ext gve a characterzato of w k+. s the set f (y k+ ). The proceedg lemma llustrates that at each terato, at each worker, we have a par of prmal decso varable Lemma 3.: For each terato k, w k+ ad a assocated subgradet (y k+, w k+ ) obtaed based o a proxmal step. Hece the m dmesoal vectors y k+, w k+ also correspods to decso varable ad subgradet par geerated based o a proxmal step at x k + cv k. B. Projecto We ext study the sequece x k, v k. Motvated by optmalty codto of problem (), we ext troduce the x followg two subspaces: A =., x =... = x, x x B =., = x = 0. We use z(a) ad z(b) to x deote the projecto of vector z oto subspaces A ad B respectvely. We observe that for ay optmal soluto x of (), frst order optmalty codtos mply that x s A ad there exsts a subgradet v F (x ), wth v B. The ext lemma qualfes the coecto betwee spaces A ad B. Lemma 3.: The spaces A ad B are orthogoal complemets. We ow ote that our algorthm x k s a projecto of decso varable y k oto space A ad v k s a projecto of subgradet w k oto the space B. These projectos are performed to gude the decso varables ad subgradets towards the approprate subspaces where the optmal solutos lve. C. Covergece Based o the prevous two sectos, we coclude that our algorthm s a combato of the proxmal method ad orthogoal projecto method. The covergece aalyss s also motvated by the oexpasve propertes of these methods. Before we proceed to the aalyss, we frst observe that by defto of w k+, we have y k+ + cw k+ = x k+ + cv k+. Therefore, oe terato of the algorthm ca be represeted as follows: x k + cv k = y k+ + cw k+ (4) x k+ = y k+ (A), v k+ = w k+ (B), (5) where w k+ s F (y k+ ) by Lemma 3.. Sce the two sequeces x k ad v k are two orthogoal spaces, ther sum has a uque orthogoal decomposto ad covergece of the sum automatcally mples covergece of x k ad v k. We wll focus o the covergece of the sum x k + cv k. We frst show that ay fxed pot of the above terato ad the set of optmal solutos to problem () are equvalet. Lemma 3.3: The vector x + cv where x A ad v B s a fxed pot of terato (5) f ad oly f x s a optmal soluto of problem () ad v s F (x). Proof: We frst assume that (x, v) s a fxed pot of terato (5). We use y, w, x +, v + to deote the updates startg from x k = x, v k = v. Sce x + cv s a fxed pot, we have x + + cv + = x + cv. Sce x, x + both are A ad v, v + are both B, by orthogoalty of A, ad B, we have x = x +, v = v +. Sce x + cv s a fxed pot, we have x + + cv + = y (A) + cw (B) = y + cw = x + cv, (6) for =,...,. We ca the sum over ad have = y (A) + cw (B) = = x+ + cv +. By costructo of x +, we have = y (A) = = x+. Therefore = w (B) = = v+. Sce v+ s B, we have = v+ = 0, whch mples that = w (B) = 0 ad w s the subspace B. Hece v + = w(b) = w = v. We combe ths wth Eq. (6), ad obta x = y = y +. Therefore, v B s also F (x) wth x A. Ths suggests that the frst order optmalty codto s satsfed ad therefore the par (x, v) s a optmal soluto ad subgradet par. Next we start from a optmal soluto ad subgradet par (x, v). We have x s A ad v s B. Sce v s F (x), we have w = v ad y = x, ad the projecto wll gve the orgal par back. Thus (x, v) s a par of fxed pot wth the compoets lyg orthogoal subspaces, whch mples that x + cv s a fxed pot of the terato.

4 By Assumpto, we have that the set of fxed pots of terato (5) s oempty. For the rest of the paper, we use x, v to deote a fxed pot of the terato (5). We ext show that the mappg from x k + cv o x k+ + cv k+ s oexpasve, whch s a key for covergece aalyss. Theorem 3.4: Let x A deote a optmal soluto of problem () ad v B be a subgradet of F (x ). The ay sequece of x k, v k, y k, w k geerated by Algorthm, we have for all k x k+ x + cv k+ cv (7) = x k x + cv k cv y k+ x k+ cw k+ cv k+ (y k+ x ) (cw k+ cv ), (y k x ) (cw k cv ) = (y k x ) (w k v ) 0. (8) = The sequece { x k x + cv k cv } k s mootocally ocreasg. Proof: We frst apply the equalty of the form z k+ z + z k z k+ +(z k z k+ ) (z k+ z ) = z k z, where z = x + cv wth the correspodg superscrpt, to x k+ + cv k+ x cv ad obta x k+ + cv k+ x cv (9) = x k + cv k x cv x k + cv k x k+ cv k+ (x k + cv k x k+ cv k+ ) (x k+ + cv k+ x cv ). The rest of the proof reles o the fact that A ad B are orthogoal complemets ad the er products betwee ay elemets of these sets are zero. For the secod term o the rght had sde of Eq. (9), we use Eq. (5) ad have x k + cv k x k+ cv k+ = y k+ + cw k+ x k+ cv k+. Sce x k+ s the projecto of y k+ A we have that y k+ x k+ les B ad smlarly cw k+ cv k+ s A. Ther er product s zero ad thus the term ca be further decomposed to x k + cv k x k+ cv k+ = y k+ x k+ + cw k+ cv k+. We ext aalyze the er product term o the rght had of Eq. (9). By Eq. (5), we have x k + cv k = y k+ + cw k+, ad thus we have x k + cv k x k+ cv k+ = yk + + cw k+ x k+ cv k+. We recall that y k+ x k+ les B ad cw k+ cv k+ s A. We also observe that x k+ x s A ad cv k+ cv s B. Combe these observatos together, we have (x k + cv k x k+ cv k+ ) (x k+ + cv k+ x cv ) = (y k+ x k+ ) (cv k+ cv ) + (cw k+ cv k+ ) (x k+ x ), = (y k+ x k+ + x k+ x ) (cv k+ cv ) + (cw k+ cv k+ ) (x k+ x + y k+ x k+ ), where the last equalty we add terms (x k+ x ) (cv k+ cv ) to the frst term ad (cw k+ cv k+ ) (y k+ x k+ ) to the secod term of the secod equalty, both of whch are zero due to the orthogoalty of A ad B. We ca ow combe the terms ad have (x k + cv k x k+ cv k+ ) (x k+ + cv k+ x cv ) (0) = (y k+ x ) (cw k+ cv ). We ca ow combe Eqs. (9)-(0) ad coclude x k+ + cv k+ x cv x k + cv k x cv = y k+ x k+ cw k+ cv k+ (y k+ x ) (cw k+ cv ). For the terms o the left had sde, we oce aga use the orthogoalty of A ad B, alog wth the fact that all x related terms are A ad v related terms are B to break dow the orm, ad have x k+ x + cv k+ cv x k x cv k cv = y k+ x k+ cw k+ cv k+ (y k+ x ) (cw k+ cv ), whch shows Eq. (7). To see that the sequece { x k x + cv k cv } k s mootocally ocreasg, we eed to show that the er product term the above le satsfes (y k+ x ) (cw k+ cv ) 0,.e., Eq. (8). We ote that sce w k+ s F (y k+ ), we have by covexty of f, (y k+ x ) (w k+ v ) 0 for =,...,. Ths establshes Eq. (8). The prevous theorem establshes that the sequece { x k x + cv k cv } k s mootocally ocreasg, we are ow equpped to show covergece of the sequece {x k } to a optmal soluto. Theorem 3.5: Let {x k } be a sequece geerated by Algorthm. The the sequece coverges to a optmal soluto of problem (). Proof: The mootocty results from prevous theorem mples that the sequece { x k x + cv k cv } k s bouded. Hece sequece {x k, v k } k has subsequet coverget sequece. We ow focus o a coverget subsequet {x kt, v kt } t ad deote ts lmt pot as x, ṽ, ad the correspodg {y kt, w kt } also coverge ad ts lmt pot as (ỹ, w). Eq. (7) suggests that x + x + cv + cv = x x + cv cv y k+ x k+ cw k+ cv k+ k= + (y k+ x ) (cw k+ cv ).

5 By Eq. (8), we have the er product term s oegatve, ad thus x x + cv cv x kt+ x cv kt+ cv + k= y k+ x k+ + cw k+ cv k+. We the take lmt as t o both sdes ad have lm x x + cv cv t x + x cv + cv lm + t k= y k+ x k+ + cw k+ cv k+. Sce the sequece {x kt, v kt } s coverget to ( x, ṽ), we have x x + cṽ cv x x cṽ cv lm + t k= y k+ x k+ + cw k+ cv k+. The left had sde s 0 ad each summad o the rght had sde s oegatve, therefore we have lm t y kt x kt = 0, lm k cw kt cv kt = 0,.e., ỹ = x, w = ṽ. Hece the pot ( x, ṽ) s a fxed pot of terato (5). We ca the use x = x, ad v = ṽ Eq. (7). Sce the value of x x + cv cṽ s gog to 0 alog the sequece ad the orgal sequece k s mootoe, we have lm k 0 x k x + cv k cṽ = 0. Therefore sequece {x k, v k } coverges. The lmt pot of ( x, ṽ) s a fxed pot of terato (5) ad thus by Lemma 3.3, x s a optmal soluto of problem (). We remarhat the above theorem guaratees covergece of the algorthm for ay stepsze choce c > 0. D. Rate of Covergece We ext show that uder some assumptos of the objectve fuctos, we ca establsh lear rate of covergece of the algorthm, ad the stepsze choce c becomes a parameter the rate of covergece. For ths secto, we assume our objectve fuctos f are cotuously dfferetable ad satsfy the followg assumpto. Assumpto : Each compoet of the objectve fucto f has Lpschtz gradet wth Lpschtz costat L ad s µ strogly covex. Note that the evet where precse values of L ad µ are mssg, a upper boud o L ad a lower boud o µ ca be used place of L ad µ for the rest of aalyss. The followg lemma relates y k+ x k+ + cw k+ cv k+ + (y k+ x ) (cw k+ cv ) to x k+ x + cv k+ cv. We later combe ths lemma wth Theorem 3.4 to show lear covergece rate. Lemma 3.6: For ay sequece of x k, v k, y k, w k geerated by Algorthm, we have that y k+ x k+ + cw k+ cv k+ () + (y k+ x ) (cw k+ cv ) { m, cµβ } x k+ x { } µ( β) + m, cv k+ cl cv for ay β (0, ). Proof: We frst focus o the er product term o the left had sde, the later combe t wth the rest of the orm terms. Sce each f s dfferetable, the vector w k+ s composed of gradet vectors,.e., w k+ = [ f (y k+ )] ad v = [ f (x )]. By strog covexty of f Assumpto, we have by [3], (y x ) ( f (y ) f (x )) µ y x, for ay y R m. For the log vector R m, we therefore have (y k+ x ) (cw k+ cv ) () = c (y k+ x ) ( f (y k+ ) f (x )) cµ = y k+ = x = cµ y k+ x. We also ote that due to the fact that each f has Lpschtz gradet wth Lpschtz costat L, we have by results [3], f (y ) f (x ) L y x, for ay y R m. Therefore, we have y k+ x = L = = y k+ x f (y k+ ) f (x ) = L w k+ v. Thus we ca troduce the factor β (0, ) to Eq. () ad have (y k+ x ) (cw k+ cv ) cµβ y k+ x + cµ( β) y k+ x cµβ y k+ x + µ( β) cl cw k+ cv. We ca ow brg the orm terms ad have y k+ x k+ + cw k+ cv k+ + (y k+ x ) (cw k+ cv ) y k+ x k+ + cµβ y k+ x + cw k+ cv k+ + µ( β) cl cw k+ cv We ext use the equalty 3 that a + b a + b, ad have y k+ x k+ + cµβ y k+ x { m, cµβ } x k+ x, 3 To see why ths equalty s true, we have a + b a b = a b 0. Therefore, [ a + b ] a b, whch mples that a + b a + b + a b = a + b.

6 cw k+ cv k+ µ( β) + cw k+ cl cv { } µ( β) m, cl cv k+ cv. By combg the prevous three equaltes, we obta Eq. (). We ext show lear rate of covergece. Theorem 3.7: For ay sequece of x k, v k geerated by Algorthm, we have that for ay β (0, ), ( { }) + m x k+ x + ( + m, cµβ { µ( β), cl }) cv k+ cv x k x + cv k cv. Proof: Recall Eqs. (7) ad () x k x + cv k cv x k+ x cv k+ cv = y k+ x k+ + cw k+ cv k+ + (y k+ x ) (cw k+ cv ), ad y k+ x k+ + cw k+ cv k+ + (y k+ x ) (cw k+ cv ) { m, cµβ } x k+ x { } µ( β) + m, cv k+ cl cv. Hece, we ca combe the prevous two les ad establsh the desred relato. The above theorem establshes lear covergece rate for the algorthm. To match the two costats, we cµβ = µ( β) β β ca set cl, ad have c = L. Ths cµβ choce of c gves = µ( β) cl = ( β)β µ L. Ths value s maxmzed at β =. We ca the have ( { + m, 4L}) [ µ x k+ x + cv k+ cv ] x k x + cv k cv. For problems wth µ L >, we have x k+ x + cv k+ cv [ x k x + cv k cv ]. For problems wth 3 µ L, we have x k+ x + cv k+ cv [ 4 x k 4+κ x + cv k cv ], where κ = µ L. We coclude that the rate of lear covergece depeds o the codto umber of the objectve fuctos. IV. CONCLUSIONS I ths paper, we propose a parallel mult-splttg proxmal method ad show that t coverges for ay postve stepsze. Whe the objectve fuctos are Lpschtz gradet ad strogly covex, the algorthm coverges learly. Future works clude exted ths algorthm to stochastc settg where delays ad errors are volved. REFERENCES [] D. P. Bertsekas. Icremetal Gradet, Subgradet, ad Proxmal Methods for Covex Optmzato: A Survey. LIDS Report 848, 00. [] D. P. Bertsekas ad J. N. Tstskls. Parallel ad Dstrbuted Computato: Numercal Methods. Athea Scetfc, Belmot, MA, 997. [3] S. Boyd, N. Parkh, E. Chu, B. Peleato, ad J. Eckste. Dstrbuted Optmzato ad Statstcal Learg va the Alteratg Drecto Method of Multplers. Foudatos ad Treds Mache Learg, 3():, 00. [4] Cahua Che, Bgsheg He, Yyu Ye, ad Xaomg Yua. The drect exteso of admm for mult-block covex mmzato problems s ot ecessarly coverget. Mathematcal Programmg, 55(-):57 79, 06. [5] Aaro Defazo, Fracs Bach, ad Smo Lacoste-Jule. Saga: A fast cremetal gradet method wth support for o-strogly covex composte objectves. I Advaces Neural Iformato Processg Systems, pages , 04. [6] J. Eckste. Augmeted Lagraga ad Alteratg Drecto Methods for Covex Optmzato: A Tutoral ad Some Illustratve Computatoal Results. Rutcor Research Report, 0. [7] Joatha Eckste ad Bear Fux Svater. Geeral projectve splttg methods for sums of maxmal mootoe operators. SIAM Joural o Cotrol ad Optmzato, 48():787 8, 009. [8] Potus Gselsso ad Stephe Boyd. Dagoal scalg douglasrachford splttg ad admm. I Decso ad Cotrol (CDC), 04 IEEE 53rd Aual Coferece o, pages IEEE, 04. [9] Re Johso ad Tog Zhag. Acceleratg stochastc gradet descet usg predctve varace reducto. I Advaces Neural Iformato Processg Systems, pages 35 33, 03. [0] J. Mota, J Xaver, P. Aguar, ad M. Püschel. ADMM For Cosesus O Colored Networks. Proceedgs of IEEE Coferece o Decso ad Cotrol (CDC), 0. [] J. Mota, J. Xaver, P. Aguar, ad M. Püschel. D-ADMM : A Commucato-Effcet Dstrbuted Algorthm For Separable Optmzato. IEEE Trasactos o Sgal Processg, 6(0):78 73, 03. [] A. Nedć ad A. Ozdaglar. Dstrbuted subgradet methods for multaget optmzato. Automatc Cotrol, IEEE Trasactos o, 54():48 6, Ja 009. [3] Yur Nesterov. Itroductory lectures o covex optmzato: A basc course, volume 87. Sprger Scece & Busess Meda, 03. [4] Neal Parkh ad Stephe P Boyd. Proxmal algorthms. Foudatos ad Treds optmzato, (3):7 39, 04. [5] I. D. Schzas, R. Rbero, ad G. B. Gaaks. Cosesus Ad Hoc WSNs wth Nosy Lks - Part I: Dstrbuted Estmato of Determstc Sgals. IEEE Trasactos o Sgal Processg, 56: , 008. [6] Mark Schmdt, Ncolas Le Roux, ad Fracs Bach. Mmzg fte sums wth the stochastc average gradet. arxv preprt arxv: , 03. [7] We Sh, Qg Lg, Gag Wu, ad Wotao Y. Extra: A exact frst-order algorthm for decetralzed cosesus optmzato. SIAM Joural o Optmzato, 5(): , 05. [8] Joatha E Spgar. Partal verse of a mootoe operator. Appled mathematcs ad optmzato, 0():47 65, 983. [9] J. N. Tstskls. Problems Decetralzed Decso Makg ad Computato. PhD thess, Massachusetts Isttute of Techology, 984. [0] E. We ad A. Ozdaglar. Dstrbuted Alteratg Drecto Method of Multplers. Proceedgs of IEEE Coferece o Decso ad Cotrol (CDC), 0. [] E. We ad A. Ozdaglar. O the O(/k) covergece of asychroous dstrbuted alteratg Drecto Method of Multplers. I Global Coferece o Sgal ad Iformato Processg (GlobalSIP), 03 IEEE, pages IEEE, 03. [] L Xao ad Tog Zhag. A proxmal stochastc gradet method wth progressve varace reducto. SIAM Joural o Optmzato, 4(4): , 04. [3] H. Zhu, A. Cao, ad G. B. Gaaks. I-Network Chael Decodg Usg Cosesus o Log-Lkelhood Rato Averages. Proceedgs of Coferece o Iformato Sceces ad Systems (CISS), 008.

arxiv: v1 [cs.lg] 22 Feb 2015

arxiv: v1 [cs.lg] 22 Feb 2015 SDCA wthout Dualty Sha Shalev-Shwartz arxv:50.0677v cs.lg Feb 05 Abstract Stochastc Dual Coordate Ascet s a popular method for solvg regularzed loss mmzato for the case of covex losses. I ths paper we

More information

LECTURE 24 LECTURE OUTLINE

LECTURE 24 LECTURE OUTLINE LECTURE 24 LECTURE OUTLINE Gradet proxmal mmzato method Noquadratc proxmal algorthms Etropy mmzato algorthm Expoetal augmeted Lagraga mehod Etropc descet algorthm **************************************

More information

Solving Constrained Flow-Shop Scheduling. Problems with Three Machines

Solving Constrained Flow-Shop Scheduling. Problems with Three Machines It J Cotemp Math Sceces, Vol 5, 2010, o 19, 921-929 Solvg Costraed Flow-Shop Schedulg Problems wth Three Maches P Pada ad P Rajedra Departmet of Mathematcs, School of Advaced Sceces, VIT Uversty, Vellore-632

More information

The Mathematical Appendix

The Mathematical Appendix The Mathematcal Appedx Defto A: If ( Λ, Ω, where ( λ λ λ whch the probablty dstrbutos,,..., Defto A. uppose that ( Λ,,..., s a expermet type, the σ-algebra o λ λ λ are defed s deoted by ( (,,...,, σ Ω.

More information

Part 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971))

Part 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971)) art 4b Asymptotc Results for MRR usg RESS Recall that the RESS statstc s a specal type of cross valdato procedure (see Alle (97)) partcular to the regresso problem ad volves fdg Y $,, the estmate at the

More information

Research Article A New Iterative Method for Common Fixed Points of a Finite Family of Nonexpansive Mappings

Research Article A New Iterative Method for Common Fixed Points of a Finite Family of Nonexpansive Mappings Hdaw Publshg Corporato Iteratoal Joural of Mathematcs ad Mathematcal Sceces Volume 009, Artcle ID 391839, 9 pages do:10.1155/009/391839 Research Artcle A New Iteratve Method for Commo Fxed Pots of a Fte

More information

Feature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture)

Feature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture) CSE 546: Mache Learg Lecture 6 Feature Selecto: Part 2 Istructor: Sham Kakade Greedy Algorthms (cotued from the last lecture) There are varety of greedy algorthms ad umerous amg covetos for these algorthms.

More information

PROJECTION PROBLEM FOR REGULAR POLYGONS

PROJECTION PROBLEM FOR REGULAR POLYGONS Joural of Mathematcal Sceces: Advaces ad Applcatos Volume, Number, 008, Pages 95-50 PROJECTION PROBLEM FOR REGULAR POLYGONS College of Scece Bejg Forestry Uversty Bejg 0008 P. R. Cha e-mal: sl@bjfu.edu.c

More information

Rademacher Complexity. Examples

Rademacher Complexity. Examples Algorthmc Foudatos of Learg Lecture 3 Rademacher Complexty. Examples Lecturer: Patrck Rebesch Verso: October 16th 018 3.1 Itroducto I the last lecture we troduced the oto of Rademacher complexty ad showed

More information

Strong Convergence of Weighted Averaged Approximants of Asymptotically Nonexpansive Mappings in Banach Spaces without Uniform Convexity

Strong Convergence of Weighted Averaged Approximants of Asymptotically Nonexpansive Mappings in Banach Spaces without Uniform Convexity BULLETIN of the MALAYSIAN MATHEMATICAL SCIENCES SOCIETY Bull. Malays. Math. Sc. Soc. () 7 (004), 5 35 Strog Covergece of Weghted Averaged Appromats of Asymptotcally Noepasve Mappgs Baach Spaces wthout

More information

Introduction to local (nonparametric) density estimation. methods

Introduction to local (nonparametric) density estimation. methods Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest

More information

Dimensionality Reduction and Learning

Dimensionality Reduction and Learning CMSC 35900 (Sprg 009) Large Scale Learg Lecture: 3 Dmesoalty Reducto ad Learg Istructors: Sham Kakade ad Greg Shakharovch L Supervsed Methods ad Dmesoalty Reducto The theme of these two lectures s that

More information

Complete Convergence and Some Maximal Inequalities for Weighted Sums of Random Variables

Complete Convergence and Some Maximal Inequalities for Weighted Sums of Random Variables Joural of Sceces, Islamc Republc of Ira 8(4): -6 (007) Uversty of Tehra, ISSN 06-04 http://sceces.ut.ac.r Complete Covergece ad Some Maxmal Iequaltes for Weghted Sums of Radom Varables M. Am,,* H.R. Nl

More information

A Remark on the Uniform Convergence of Some Sequences of Functions

A Remark on the Uniform Convergence of Some Sequences of Functions Advaces Pure Mathematcs 05 5 57-533 Publshed Ole July 05 ScRes. http://www.scrp.org/joural/apm http://dx.do.org/0.436/apm.05.59048 A Remark o the Uform Covergece of Some Sequeces of Fuctos Guy Degla Isttut

More information

A tighter lower bound on the circuit size of the hardest Boolean functions

A tighter lower bound on the circuit size of the hardest Boolean functions Electroc Colloquum o Computatoal Complexty, Report No. 86 2011) A tghter lower boud o the crcut sze of the hardest Boolea fuctos Masak Yamamoto Abstract I [IPL2005], Fradse ad Mlterse mproved bouds o the

More information

MATH 247/Winter Notes on the adjoint and on normal operators.

MATH 247/Winter Notes on the adjoint and on normal operators. MATH 47/Wter 00 Notes o the adjot ad o ormal operators I these otes, V s a fte dmesoal er product space over, wth gve er * product uv, T, S, T, are lear operators o V U, W are subspaces of V Whe we say

More information

Chapter 5 Properties of a Random Sample

Chapter 5 Properties of a Random Sample Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample

More information

Aitken delta-squared generalized Juncgk-type iterative procedure

Aitken delta-squared generalized Juncgk-type iterative procedure Atke delta-squared geeralzed Jucgk-type teratve procedure M. De la Se Isttute of Research ad Developmet of Processes. Uversty of Basque Coutry Campus of Leoa (Bzkaa) PO Box. 644- Blbao, 488- Blbao. SPAIN

More information

Likewise, properties of the optimal policy for equipment replacement & maintenance problems can be used to reduce the computation.

Likewise, properties of the optimal policy for equipment replacement & maintenance problems can be used to reduce the computation. Whe solvg a vetory repleshmet problem usg a MDP model, kowg that the optmal polcy s of the form (s,s) ca reduce the computatoal burde. That s, f t s optmal to replesh the vetory whe the vetory level s,

More information

Functions of Random Variables

Functions of Random Variables Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,

More information

TESTS BASED ON MAXIMUM LIKELIHOOD

TESTS BASED ON MAXIMUM LIKELIHOOD ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal

More information

ECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity

ECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity ECONOMETRIC THEORY MODULE VIII Lecture - 6 Heteroskedastcty Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur . Breusch Paga test Ths test ca be appled whe the replcated data

More information

1 Convergence of the Arnoldi method for eigenvalue problems

1 Convergence of the Arnoldi method for eigenvalue problems Lecture otes umercal lear algebra Arold method covergece Covergece of the Arold method for egevalue problems Recall that, uless t breaks dow, k steps of the Arold method geerates a orthogoal bass of a

More information

1 Lyapunov Stability Theory

1 Lyapunov Stability Theory Lyapuov Stablty heory I ths secto we cosder proofs of stablty of equlbra of autoomous systems. hs s stadard theory for olear systems, ad oe of the most mportat tools the aalyss of olear systems. It may

More information

Support vector machines

Support vector machines CS 75 Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Outle Outle: Algorthms for lear decso boudary Support vector maches Mamum marg hyperplae.

More information

Point Estimation: definition of estimators

Point Estimation: definition of estimators Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.

More information

Analysis of Lagrange Interpolation Formula

Analysis of Lagrange Interpolation Formula P IJISET - Iteratoal Joural of Iovatve Scece, Egeerg & Techology, Vol. Issue, December 4. www.jset.com ISS 348 7968 Aalyss of Lagrage Iterpolato Formula Vjay Dahya PDepartmet of MathematcsMaharaja Surajmal

More information

A Study on Generalized Generalized Quasi hyperbolic Kac Moody algebra QHGGH of rank 10

A Study on Generalized Generalized Quasi hyperbolic Kac Moody algebra QHGGH of rank 10 Global Joural of Mathematcal Sceces: Theory ad Practcal. ISSN 974-3 Volume 9, Number 3 (7), pp. 43-4 Iteratoal Research Publcato House http://www.rphouse.com A Study o Geeralzed Geeralzed Quas (9) hyperbolc

More information

Research on SVM Prediction Model Based on Chaos Theory

Research on SVM Prediction Model Based on Chaos Theory Advaced Scece ad Techology Letters Vol.3 (SoftTech 06, pp.59-63 http://dx.do.org/0.457/astl.06.3.3 Research o SVM Predcto Model Based o Chaos Theory Sog Lagog, Wu Hux, Zhag Zezhog 3, College of Iformato

More information

CS286.2 Lecture 4: Dinur s Proof of the PCP Theorem

CS286.2 Lecture 4: Dinur s Proof of the PCP Theorem CS86. Lecture 4: Dur s Proof of the PCP Theorem Scrbe: Thom Bohdaowcz Prevously, we have prove a weak verso of the PCP theorem: NP PCP 1,1/ (r = poly, q = O(1)). Wth ths result we have the desred costat

More information

Cubic Nonpolynomial Spline Approach to the Solution of a Second Order Two-Point Boundary Value Problem

Cubic Nonpolynomial Spline Approach to the Solution of a Second Order Two-Point Boundary Value Problem Joural of Amerca Scece ;6( Cubc Nopolyomal Sple Approach to the Soluto of a Secod Order Two-Pot Boudary Value Problem W.K. Zahra, F.A. Abd El-Salam, A.A. El-Sabbagh ad Z.A. ZAk * Departmet of Egeerg athematcs

More information

Bayes (Naïve or not) Classifiers: Generative Approach

Bayes (Naïve or not) Classifiers: Generative Approach Logstc regresso Bayes (Naïve or ot) Classfers: Geeratve Approach What do we mea by Geeratve approach: Lear p(y), p(x y) ad the apply bayes rule to compute p(y x) for makg predctos Ths s essetally makg

More information

Chapter 9 Jordan Block Matrices

Chapter 9 Jordan Block Matrices Chapter 9 Jorda Block atrces I ths chapter we wll solve the followg problem. Gve a lear operator T fd a bass R of F such that the matrx R (T) s as smple as possble. f course smple s a matter of taste.

More information

BERNSTEIN COLLOCATION METHOD FOR SOLVING NONLINEAR DIFFERENTIAL EQUATIONS. Aysegul Akyuz Dascioglu and Nese Isler

BERNSTEIN COLLOCATION METHOD FOR SOLVING NONLINEAR DIFFERENTIAL EQUATIONS. Aysegul Akyuz Dascioglu and Nese Isler Mathematcal ad Computatoal Applcatos, Vol. 8, No. 3, pp. 293-300, 203 BERNSTEIN COLLOCATION METHOD FOR SOLVING NONLINEAR DIFFERENTIAL EQUATIONS Aysegul Ayuz Dascoglu ad Nese Isler Departmet of Mathematcs,

More information

Summary of the lecture in Biostatistics

Summary of the lecture in Biostatistics Summary of the lecture Bostatstcs Probablty Desty Fucto For a cotuos radom varable, a probablty desty fucto s a fucto such that: 0 dx a b) b a dx A probablty desty fucto provdes a smple descrpto of the

More information

A New Method for Decision Making Based on Soft Matrix Theory

A New Method for Decision Making Based on Soft Matrix Theory Joural of Scetfc esearch & eports 3(5): 0-7, 04; rtcle o. JS.04.5.00 SCIENCEDOMIN teratoal www.scecedoma.org New Method for Decso Mag Based o Soft Matrx Theory Zhmg Zhag * College of Mathematcs ad Computer

More information

Runtime analysis RLS on OneMax. Heuristic Optimization

Runtime analysis RLS on OneMax. Heuristic Optimization Lecture 6 Rutme aalyss RLS o OeMax trals of {,, },, l ( + ɛ) l ( ɛ)( ) l Algorthm Egeerg Group Hasso Platter Isttute, Uversty of Potsdam 9 May T, We wat to rgorously uderstad ths behavor 9 May / Rutme

More information

CS 2750 Machine Learning. Lecture 8. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x

CS 2750 Machine Learning. Lecture 8. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x CS 75 Mache Learg Lecture 8 Lear regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear combato of put compoets f + + + K d d K k - parameters

More information

Block-Based Compact Thermal Modeling of Semiconductor Integrated Circuits

Block-Based Compact Thermal Modeling of Semiconductor Integrated Circuits Block-Based Compact hermal Modelg of Semcoductor Itegrated Crcuts Master s hess Defese Caddate: Jg Ba Commttee Members: Dr. Mg-Cheg Cheg Dr. Daqg Hou Dr. Robert Schllg July 27, 2009 Outle Itroducto Backgroud

More information

A new type of optimization method based on conjugate directions

A new type of optimization method based on conjugate directions A ew type of optmzato method based o cojugate drectos Pa X Scece School aj Uversty of echology ad Educato (UE aj Cha e-mal: pax94@sacom Abstract A ew type of optmzato method based o cojugate drectos s

More information

Consensus Control for a Class of High Order System via Sliding Mode Control

Consensus Control for a Class of High Order System via Sliding Mode Control Cosesus Cotrol for a Class of Hgh Order System va Sldg Mode Cotrol Chagb L, Y He, ad Aguo Wu School of Electrcal ad Automato Egeerg, Taj Uversty, Taj, Cha, 300072 Abstract. I ths paper, cosesus problem

More information

Mu Sequences/Series Solutions National Convention 2014

Mu Sequences/Series Solutions National Convention 2014 Mu Sequeces/Seres Solutos Natoal Coveto 04 C 6 E A 6C A 6 B B 7 A D 7 D C 7 A B 8 A B 8 A C 8 E 4 B 9 B 4 E 9 B 4 C 9 E C 0 A A 0 D B 0 C C Usg basc propertes of arthmetc sequeces, we fd a ad bm m We eed

More information

Lecture 3 Probability review (cont d)

Lecture 3 Probability review (cont d) STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto

More information

Assignment 5/MATH 247/Winter Due: Friday, February 19 in class (!) (answers will be posted right after class)

Assignment 5/MATH 247/Winter Due: Friday, February 19 in class (!) (answers will be posted right after class) Assgmet 5/MATH 7/Wter 00 Due: Frday, February 9 class (!) (aswers wll be posted rght after class) As usual, there are peces of text, before the questos [], [], themselves. Recall: For the quadratc form

More information

Lecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model

Lecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The

More information

Discrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b

Discrete Mathematics and Probability Theory Fall 2016 Seshia and Walrand DIS 10b CS 70 Dscrete Mathematcs ad Probablty Theory Fall 206 Sesha ad Walrad DIS 0b. Wll I Get My Package? Seaky delvery guy of some compay s out delverg packages to customers. Not oly does he had a radom package

More information

PTAS for Bin-Packing

PTAS for Bin-Packing CS 663: Patter Matchg Algorthms Scrbe: Che Jag /9/00. Itroducto PTAS for B-Packg The B-Packg problem s NP-hard. If we use approxmato algorthms, the B-Packg problem could be solved polyomal tme. For example,

More information

Lecture 16: Backpropogation Algorithm Neural Networks with smooth activation functions

Lecture 16: Backpropogation Algorithm Neural Networks with smooth activation functions CO-511: Learg Theory prg 2017 Lecturer: Ro Lv Lecture 16: Bacpropogato Algorthm Dsclamer: These otes have ot bee subected to the usual scruty reserved for formal publcatos. They may be dstrbuted outsde

More information

CIS 800/002 The Algorithmic Foundations of Data Privacy October 13, Lecture 9. Database Update Algorithms: Multiplicative Weights

CIS 800/002 The Algorithmic Foundations of Data Privacy October 13, Lecture 9. Database Update Algorithms: Multiplicative Weights CIS 800/002 The Algorthmc Foudatos of Data Prvacy October 13, 2011 Lecturer: Aaro Roth Lecture 9 Scrbe: Aaro Roth Database Update Algorthms: Multplcatve Weghts We ll recall aga) some deftos from last tme:

More information

Generalization of the Dissimilarity Measure of Fuzzy Sets

Generalization of the Dissimilarity Measure of Fuzzy Sets Iteratoal Mathematcal Forum 2 2007 o. 68 3395-3400 Geeralzato of the Dssmlarty Measure of Fuzzy Sets Faramarz Faghh Boformatcs Laboratory Naobotechology Research Ceter vesa Research Isttute CECR Tehra

More information

Investigation of Partially Conditional RP Model with Response Error. Ed Stanek

Investigation of Partially Conditional RP Model with Response Error. Ed Stanek Partally Codtoal Radom Permutato Model 7- vestgato of Partally Codtoal RP Model wth Respose Error TRODUCTO Ed Staek We explore the predctor that wll result a smple radom sample wth respose error whe a

More information

Research Article A New Derivation and Recursive Algorithm Based on Wronskian Matrix for Vandermonde Inverse Matrix

Research Article A New Derivation and Recursive Algorithm Based on Wronskian Matrix for Vandermonde Inverse Matrix Mathematcal Problems Egeerg Volume 05 Artcle ID 94757 7 pages http://ddoorg/055/05/94757 Research Artcle A New Dervato ad Recursve Algorthm Based o Wroska Matr for Vadermode Iverse Matr Qu Zhou Xja Zhag

More information

MULTIDIMENSIONAL HETEROGENEOUS VARIABLE PREDICTION BASED ON EXPERTS STATEMENTS. Gennadiy Lbov, Maxim Gerasimov

MULTIDIMENSIONAL HETEROGENEOUS VARIABLE PREDICTION BASED ON EXPERTS STATEMENTS. Gennadiy Lbov, Maxim Gerasimov Iteratoal Boo Seres "Iformato Scece ad Computg" 97 MULTIIMNSIONAL HTROGNOUS VARIABL PRICTION BAS ON PRTS STATMNTS Geady Lbov Maxm Gerasmov Abstract: I the wors [ ] we proposed a approach of formg a cosesus

More information

arxiv:math/ v1 [math.gm] 8 Dec 2005

arxiv:math/ v1 [math.gm] 8 Dec 2005 arxv:math/05272v [math.gm] 8 Dec 2005 A GENERALIZATION OF AN INEQUALITY FROM IMO 2005 NIKOLAI NIKOLOV The preset paper was spred by the thrd problem from the IMO 2005. A specal award was gve to Yure Boreko

More information

13. Parametric and Non-Parametric Uncertainties, Radial Basis Functions and Neural Network Approximations

13. Parametric and Non-Parametric Uncertainties, Radial Basis Functions and Neural Network Approximations Lecture 7 3. Parametrc ad No-Parametrc Ucertates, Radal Bass Fuctos ad Neural Network Approxmatos he parameter estmato algorthms descrbed prevous sectos were based o the assumpto that the system ucertates

More information

Simulation Output Analysis

Simulation Output Analysis Smulato Output Aalyss Summary Examples Parameter Estmato Sample Mea ad Varace Pot ad Iterval Estmato ermatg ad o-ermatg Smulato Mea Square Errors Example: Sgle Server Queueg System x(t) S 4 S 4 S 3 S 5

More information

Comparing Different Estimators of three Parameters for Transmuted Weibull Distribution

Comparing Different Estimators of three Parameters for Transmuted Weibull Distribution Global Joural of Pure ad Appled Mathematcs. ISSN 0973-768 Volume 3, Number 9 (207), pp. 55-528 Research Ida Publcatos http://www.rpublcato.com Comparg Dfferet Estmators of three Parameters for Trasmuted

More information

Bayes Estimator for Exponential Distribution with Extension of Jeffery Prior Information

Bayes Estimator for Exponential Distribution with Extension of Jeffery Prior Information Malaysa Joural of Mathematcal Sceces (): 97- (9) Bayes Estmator for Expoetal Dstrbuto wth Exteso of Jeffery Pror Iformato Hadeel Salm Al-Kutub ad Noor Akma Ibrahm Isttute for Mathematcal Research, Uverst

More information

( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model

( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch

More information

A conic cutting surface method for linear-quadraticsemidefinite

A conic cutting surface method for linear-quadraticsemidefinite A coc cuttg surface method for lear-quadratcsemdefte programmg Mohammad R. Osoorouch Calfora State Uversty Sa Marcos Sa Marcos, CA Jot wor wth Joh E. Mtchell RPI July 3, 2008 Outle: Secod-order coe: defto

More information

Non-uniform Turán-type problems

Non-uniform Turán-type problems Joural of Combatoral Theory, Seres A 111 2005 106 110 wwwelsevercomlocatecta No-uform Turá-type problems DhruvMubay 1, Y Zhao 2 Departmet of Mathematcs, Statstcs, ad Computer Scece, Uversty of Illos at

More information

X ε ) = 0, or equivalently, lim

X ε ) = 0, or equivalently, lim Revew for the prevous lecture Cocepts: order statstcs Theorems: Dstrbutos of order statstcs Examples: How to get the dstrbuto of order statstcs Chapter 5 Propertes of a Radom Sample Secto 55 Covergece

More information

Research Article Some Strong Limit Theorems for Weighted Product Sums of ρ-mixing Sequences of Random Variables

Research Article Some Strong Limit Theorems for Weighted Product Sums of ρ-mixing Sequences of Random Variables Hdaw Publshg Corporato Joural of Iequaltes ad Applcatos Volume 2009, Artcle ID 174768, 10 pages do:10.1155/2009/174768 Research Artcle Some Strog Lmt Theorems for Weghted Product Sums of ρ-mxg Sequeces

More information

Uniform asymptotical stability of almost periodic solution of a discrete multispecies Lotka-Volterra competition system

Uniform asymptotical stability of almost periodic solution of a discrete multispecies Lotka-Volterra competition system Iteratoal Joural of Egeerg ad Advaced Research Techology (IJEART) ISSN: 2454-9290, Volume-2, Issue-1, Jauary 2016 Uform asymptotcal stablty of almost perodc soluto of a dscrete multspeces Lotka-Volterra

More information

Application of Calibration Approach for Regression Coefficient Estimation under Two-stage Sampling Design

Application of Calibration Approach for Regression Coefficient Estimation under Two-stage Sampling Design Authors: Pradp Basak, Kaustav Adtya, Hukum Chadra ad U.C. Sud Applcato of Calbrato Approach for Regresso Coeffcet Estmato uder Two-stage Samplg Desg Pradp Basak, Kaustav Adtya, Hukum Chadra ad U.C. Sud

More information

CHAPTER VI Statistical Analysis of Experimental Data

CHAPTER VI Statistical Analysis of Experimental Data Chapter VI Statstcal Aalyss of Expermetal Data CHAPTER VI Statstcal Aalyss of Expermetal Data Measuremets do ot lead to a uque value. Ths s a result of the multtude of errors (maly radom errors) that ca

More information

Estimation of Stress- Strength Reliability model using finite mixture of exponential distributions

Estimation of Stress- Strength Reliability model using finite mixture of exponential distributions Iteratoal Joural of Computatoal Egeerg Research Vol, 0 Issue, Estmato of Stress- Stregth Relablty model usg fte mxture of expoetal dstrbutos K.Sadhya, T.S.Umamaheswar Departmet of Mathematcs, Lal Bhadur

More information

Supervised learning: Linear regression Logistic regression

Supervised learning: Linear regression Logistic regression CS 57 Itroducto to AI Lecture 4 Supervsed learg: Lear regresso Logstc regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Data: D { D D.. D D Supervsed learg d a set of eamples s

More information

Convergence of the Desroziers scheme and its relation to the lag innovation diagnostic

Convergence of the Desroziers scheme and its relation to the lag innovation diagnostic Covergece of the Desrozers scheme ad ts relato to the lag ovato dagostc chard Méard Evromet Caada, Ar Qualty esearch Dvso World Weather Ope Scece Coferece Motreal, August 9, 04 o t t O x x x y x y Oservato

More information

Communication-Efficient Distributed Primal-Dual Algorithm for Saddle Point Problems

Communication-Efficient Distributed Primal-Dual Algorithm for Saddle Point Problems Commucato-Effcet Dstrbuted Prmal-Dual Algorthm for Saddle Pot Problems Yaodog Yu Nayag Techologcal Uversty ydyu@tu.edu.sg Sul Lu Nayag Techologcal Uversty lusl@tu.edu.sg So Jal Pa Nayag Techologcal Uversty

More information

An Introduction to. Support Vector Machine

An Introduction to. Support Vector Machine A Itroducto to Support Vector Mache Support Vector Mache (SVM) A classfer derved from statstcal learg theory by Vapk, et al. 99 SVM became famous whe, usg mages as put, t gave accuracy comparable to eural-etwork

More information

UNIT 2 SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS

UNIT 2 SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS Numercal Computg -I UNIT SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS Structure Page Nos..0 Itroducto 6. Objectves 7. Ital Approxmato to a Root 7. Bsecto Method 8.. Error Aalyss 9.4 Regula Fals Method

More information

1 Onto functions and bijections Applications to Counting

1 Onto functions and bijections Applications to Counting 1 Oto fuctos ad bectos Applcatos to Coutg Now we move o to a ew topc. Defto 1.1 (Surecto. A fucto f : A B s sad to be surectve or oto f for each b B there s some a A so that f(a B. What are examples of

More information

A New Family of Transformations for Lifetime Data

A New Family of Transformations for Lifetime Data Proceedgs of the World Cogress o Egeerg 4 Vol I, WCE 4, July - 4, 4, Lodo, U.K. A New Famly of Trasformatos for Lfetme Data Lakhaa Watthaacheewakul Abstract A famly of trasformatos s the oe of several

More information

5 Short Proofs of Simplified Stirling s Approximation

5 Short Proofs of Simplified Stirling s Approximation 5 Short Proofs of Smplfed Strlg s Approxmato Ofr Gorodetsky, drtymaths.wordpress.com Jue, 20 0 Itroducto Strlg s approxmato s the followg (somewhat surprsg) approxmato of the factoral,, usg elemetary fuctos:

More information

Lebesgue Measure of Generalized Cantor Set

Lebesgue Measure of Generalized Cantor Set Aals of Pure ad Appled Mathematcs Vol., No.,, -8 ISSN: -8X P), -888ole) Publshed o 8 May www.researchmathsc.org Aals of Lebesgue Measure of Geeralzed ator Set Md. Jahurul Islam ad Md. Shahdul Islam Departmet

More information

Analysis of Variance with Weibull Data

Analysis of Variance with Weibull Data Aalyss of Varace wth Webull Data Lahaa Watthaacheewaul Abstract I statstcal data aalyss by aalyss of varace, the usual basc assumptos are that the model s addtve ad the errors are radomly, depedetly, ad

More information

On Modified Interval Symmetric Single-Step Procedure ISS2-5D for the Simultaneous Inclusion of Polynomial Zeros

On Modified Interval Symmetric Single-Step Procedure ISS2-5D for the Simultaneous Inclusion of Polynomial Zeros It. Joural of Math. Aalyss, Vol. 7, 2013, o. 20, 983-988 HIKARI Ltd, www.m-hkar.com O Modfed Iterval Symmetrc Sgle-Step Procedure ISS2-5D for the Smultaeous Icluso of Polyomal Zeros 1 Nora Jamalud, 1 Masor

More information

Lecture Note to Rice Chapter 8

Lecture Note to Rice Chapter 8 ECON 430 HG revsed Nov 06 Lecture Note to Rce Chapter 8 Radom matrces Let Y, =,,, m, =,,, be radom varables (r.v. s). The matrx Y Y Y Y Y Y Y Y Y Y = m m m s called a radom matrx ( wth a ot m-dmesoal dstrbuto,

More information

An Accelerated Proximal Coordinate Gradient Method

An Accelerated Proximal Coordinate Gradient Method A Accelerated Proxmal Coordate Gradet Method Qhag L Uversty of Iowa Iowa Cty IA USA qhag-l@uowaedu Zhaosog Lu Smo Fraser Uversty Buraby BC Caada zhaosog@sfuca L Xao Mcrosoft Research Redmod WA USA lxao@mcrosoftcom

More information

Kernel-based Methods and Support Vector Machines

Kernel-based Methods and Support Vector Machines Kerel-based Methods ad Support Vector Maches Larr Holder CptS 570 Mache Learg School of Electrcal Egeerg ad Computer Scece Washgto State Uverst Refereces Muller et al. A Itroducto to Kerel-Based Learg

More information

Lecture 07: Poles and Zeros

Lecture 07: Poles and Zeros Lecture 07: Poles ad Zeros Defto of poles ad zeros The trasfer fucto provdes a bass for determg mportat system respose characterstcs wthout solvg the complete dfferetal equato. As defed, the trasfer fucto

More information

THE PROBABILISTIC STABILITY FOR THE GAMMA FUNCTIONAL EQUATION

THE PROBABILISTIC STABILITY FOR THE GAMMA FUNCTIONAL EQUATION Joural of Scece ad Arts Year 12, No. 3(2), pp. 297-32, 212 ORIGINAL AER THE ROBABILISTIC STABILITY FOR THE GAMMA FUNCTIONAL EQUATION DOREL MIHET 1, CLAUDIA ZAHARIA 1 Mauscrpt receved: 3.6.212; Accepted

More information

Ordinary Least Squares Regression. Simple Regression. Algebra and Assumptions.

Ordinary Least Squares Regression. Simple Regression. Algebra and Assumptions. Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos

More information

4 Inner Product Spaces

4 Inner Product Spaces 11.MH1 LINEAR ALGEBRA Summary Notes 4 Ier Product Spaces Ier product s the abstracto to geeral vector spaces of the famlar dea of the scalar product of two vectors or 3. I what follows, keep these key

More information

3D Geometry for Computer Graphics. Lesson 2: PCA & SVD

3D Geometry for Computer Graphics. Lesson 2: PCA & SVD 3D Geometry for Computer Graphcs Lesso 2: PCA & SVD Last week - egedecomposto We wat to lear how the matrx A works: A 2 Last week - egedecomposto If we look at arbtrary vectors, t does t tell us much.

More information

Homework 1: Solutions Sid Banerjee Problem 1: (Practice with Asymptotic Notation) ORIE 4520: Stochastics at Scale Fall 2015

Homework 1: Solutions Sid Banerjee Problem 1: (Practice with Asymptotic Notation) ORIE 4520: Stochastics at Scale Fall 2015 Fall 05 Homework : Solutos Problem : (Practce wth Asymptotc Notato) A essetal requremet for uderstadg scalg behavor s comfort wth asymptotc (or bg-o ) otato. I ths problem, you wll prove some basc facts

More information

Analyzing Fuzzy System Reliability Using Vague Set Theory

Analyzing Fuzzy System Reliability Using Vague Set Theory Iteratoal Joural of Appled Scece ad Egeerg 2003., : 82-88 Aalyzg Fuzzy System Relablty sg Vague Set Theory Shy-Mg Che Departmet of Computer Scece ad Iformato Egeerg, Natoal Tawa versty of Scece ad Techology,

More information

Entropy ISSN by MDPI

Entropy ISSN by MDPI Etropy 2003, 5, 233-238 Etropy ISSN 1099-4300 2003 by MDPI www.mdp.org/etropy O the Measure Etropy of Addtve Cellular Automata Hasa Aı Arts ad Sceces Faculty, Departmet of Mathematcs, Harra Uversty; 63100,

More information

An Indian Journal FULL PAPER ABSTRACT KEYWORDS. Trade Science Inc. Research on scheme evaluation method of automation mechatronic systems

An Indian Journal FULL PAPER ABSTRACT KEYWORDS. Trade Science Inc. Research on scheme evaluation method of automation mechatronic systems [ype text] [ype text] [ype text] ISSN : 0974-7435 Volume 0 Issue 6 Boechology 204 Ida Joural FULL PPER BIJ, 0(6, 204 [927-9275] Research o scheme evaluato method of automato mechatroc systems BSRC Che

More information

{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:

{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution: Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed

More information

Unimodality Tests for Global Optimization of Single Variable Functions Using Statistical Methods

Unimodality Tests for Global Optimization of Single Variable Functions Using Statistical Methods Malaysa Umodalty Joural Tests of Mathematcal for Global Optmzato Sceces (): of 05 Sgle - 5 Varable (007) Fuctos Usg Statstcal Methods Umodalty Tests for Global Optmzato of Sgle Varable Fuctos Usg Statstcal

More information

0/1 INTEGER PROGRAMMING AND SEMIDEFINTE PROGRAMMING

0/1 INTEGER PROGRAMMING AND SEMIDEFINTE PROGRAMMING CONVEX OPIMIZAION AND INERIOR POIN MEHODS FINAL PROJEC / INEGER PROGRAMMING AND SEMIDEFINE PROGRAMMING b Luca Buch ad Natala Vktorova CONENS:.Itroducto.Formulato.Applcato to Kapsack Problem 4.Cuttg Plaes

More information

The number of observed cases The number of parameters. ith case of the dichotomous dependent variable. the ith case of the jth parameter

The number of observed cases The number of parameters. ith case of the dichotomous dependent variable. the ith case of the jth parameter LOGISTIC REGRESSION Notato Model Logstc regresso regresses a dchotomous depedet varable o a set of depedet varables. Several methods are mplemeted for selectg the depedet varables. The followg otato s

More information

Investigating Cellular Automata

Investigating Cellular Automata Researcher: Taylor Dupuy Advsor: Aaro Wootto Semester: Fall 4 Ivestgatg Cellular Automata A Overvew of Cellular Automata: Cellular Automata are smple computer programs that geerate rows of black ad whte

More information

Complete Convergence for Weighted Sums of Arrays of Rowwise Asymptotically Almost Negative Associated Random Variables

Complete Convergence for Weighted Sums of Arrays of Rowwise Asymptotically Almost Negative Associated Random Variables A^VÇÚO 1 32 ò 1 5 Ï 2016 c 10 Chese Joural of Appled Probablty ad Statstcs Oct., 2016, Vol. 32, No. 5, pp. 489-498 do: 10.3969/j.ss.1001-4268.2016.05.005 Complete Covergece for Weghted Sums of Arrays of

More information

Multivariate Transformation of Variables and Maximum Likelihood Estimation

Multivariate Transformation of Variables and Maximum Likelihood Estimation Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty

More information

arxiv: v4 [math.nt] 14 Aug 2015

arxiv: v4 [math.nt] 14 Aug 2015 arxv:52.799v4 [math.nt] 4 Aug 25 O the propertes of terated bomal trasforms for the Padova ad Perr matrx sequeces Nazmye Ylmaz ad Necat Tasara Departmet of Mathematcs, Faculty of Scece, Selcu Uversty,

More information

Nonlinear Piecewise-Defined Difference Equations with Reciprocal Quadratic Terms

Nonlinear Piecewise-Defined Difference Equations with Reciprocal Quadratic Terms Joural of Matematcs ad Statstcs Orgal Researc Paper Nolear Pecewse-Defed Dfferece Equatos wt Recprocal Quadratc Terms Ramada Sabra ad Saleem Safq Al-Asab Departmet of Matematcs, Faculty of Scece, Jaza

More information

Algorithms Theory, Solution for Assignment 2

Algorithms Theory, Solution for Assignment 2 Juor-Prof. Dr. Robert Elsässer, Marco Muñz, Phllp Hedegger WS 2009/200 Algorthms Theory, Soluto for Assgmet 2 http://lak.formatk.u-freburg.de/lak_teachg/ws09_0/algo090.php Exercse 2. - Fast Fourer Trasform

More information