Decentralized Double Stochastic Averaging Gradient

Size: px
Start display at page:

Download "Decentralized Double Stochastic Averaging Gradient"

Transcription

1 Deceralized Double Sochasic Averagig Gradie Arya Mokhari ad Alejadro Ribeiro Deparme of Elecrical ad Sysems Egieerig, Uiversiy of Pesylvaia Absrac This paper cosiders cove opimizaio problems where odes of a ework have access o summads of a global objecive fucio. Each of hese local objecives is furher assumed o be a average of a fiie se of fucios. The moivaio for his seup is o solve large scale machie learig problems where elemes of he raiig se are disribued o muliple compuaioal elemes. The deceralized double sochasic averagig gradie (DSA) algorihm is proposed as a soluio aleraive ha relies o: (i) The use of local sochasic averagig gradies isead of local full gradies. (ii) Deermiaio of desce seps as differeces of cosecuive sochasic averagig gradies. The algorihm is show o approach he opimal argume a a liear rae. This is i coras o all oher available mehods for disribued sochasic opimizaio ha coverge a subliear raes. Numerical eperimes verify liear covergece of DSA ad illusrae is advaages relaive o hese oher aleraives. I. INTRODUCTION Cosider a variable R p ad a coeced ework of size N where each ode has access o a local objecive fucio f : R p R. The local objecive fucio f () is defied as he average of fucios f,i () ha ca be idividually evaluaed a ode. Ages cooperae o solve he global opimizaio := argmi f () = argmi q i= f,i (). () The formulaio i () models large scale machie learig problems where elemes of he raiig se are disribued o muliple compuaioal elemes ad represes a opimal classifier, 2. Aalogous formulaios are also of ieres i deceralized corol 3, 4 ad sesor eworks 5 7. Our ieres here is i solvig () wih a mehod ha is disribued odes operae o heir local fucios ad commuicae wih eighbors ad sochasic odes uilize oly oe ou of he fucios f,i o deermie a desce direcio. Disribued mehods germae o his paper are deceralized gradie desce (DGD) ad is varias 8, as well as he eac firs order algorihm (EXTRA) 2. A issue wih he former ha is solved by he laer is he lack of liear covergece rae guaraees ha EXTRA achieves by usig ieraios ha rely o iformaio of wo cosecuive seps. Sochasic opimizaio mehods relaed o he proposal i his paper are sochasic gradie desce 3 6 ad sochasic averagig gradie mehods 7, 8. As i he case of disribued opimizaio, he former have subliear covergece raes bu he laer have liear covergece raes. They achieve hese liear raes by usig icremeal gradies o reduce he sochasic gradie oise. This reducio follows from a memory rade ha permis maiaiig a average of pas gradies i which oly oe erm is updaed per ieraio. The coribuio of his paper is o develop he deceralized double sochasic averagig gradie (DSA) mehod, a ovel deceralized sochasic algorihm for solvig (). The mehod eplois a ew ierpreaio of EXTRA as a saddle poi mehod (Secio II) ad uses sochasic averagig gradies i lieu of gradies (Secio III). The proposed mehod coverges liearly o he opimal argume i epecaio (Secio IV). This is i coras o all oher disribued sochasic mehods o solve () ha coverge a subliear raes. Numerical resuls verify ha DSA is he oly sochasic deceralized algorihm wih liear covergece rae (Secio V). Proofs of resuls i his paper are available i 9. II. DECENTRALIZED DOUBLE GRADIENT DESCENT Cosider a coeced ework ha coais N odes such ha each ode ca oly commuicae wih odes i is eighborhood N. Defie R p as a local copy of he variable ha is kep a ode. I deceralized opimizaio, odes ry o miimize heir local fucios f ( ) while esurig ha heir local variables coicide wih he variables m of all eighbors m N which, give ha he ework is coeced, esures ha he variables of all odes are he same ad reders he problem equivale o (). DGD is a well kow mehod for deceralized opimizaio ha relies o he iroducio of oegaive weighs w ij 0 ha are o ull if ad oly if m = or if m N. Leig N be a discree ime ide ad α a give sepsize, DGD is defied by he recursio + = w m m α f ( ), =,..., N. (2) m= Sice w m = 0 whe m ad m / N, i follows from (2) ha ode updaes by performig a average over he variables m of is eighbors m N ad is ow, followed by desce hrough he egaive local gradie f ( ). If a cosa sepsize is used, DGD ieraes approach a eighborhood of he opimal argume of () bu do coverge eacly. To achieve eac covergece dimiishig sepsizes are used bu he resulig covergece rae is subliear 8. EXTRA resolves eiher of hese issues by miig wo cosecuive DGD ieraios wih differe weigh marices ad opposie sigs. To be precise, iroduce a secod se of weighs w m wih he same properies as he weighs w m ad defie EXTRA hrough he recursio + = + w m m m= m= w m m (3) α f ( ) f ( ), =,..., N. Observe ha (3) is well defied for > 0. For = 0 we uilize he regular DGD ieraio i (2). I he omeclaure of his paper we say ha EXTRA performs a deceralized double gradie desce sep because i operaes i deceralized maer while uilizig a differece of wo gradies as desce direcio. Mior modificaio as i is, he use of his gradie differece i lieu of

2 simple gradies edows era wih eac liear covergece o he opimal argume 2. To udersad he raioaliy behid he EXTRA updae, we defie marices ad vecors o rewrie updaes i (3) for differe odes as a sigle equaio. To do so, defie he vecor := ;... ; N R Np which cocaeaes he local ieraes, ad he aggregae fucio f : R Np R as f () = f(,..., N ) := f ( ). (4) Moreover, Cosider he marices W R N N ad W R N N formed by compoes w m ad w m, respecively. Defie he marices Z := W I R Np Np ad Z := W I R Np Np as he Kroecker producs of he weigh marices W R N N ad W R N N by he ideiy mari I R p p, respecively. Cosiderig hese defiiios, we ca rewrie he EXTRA s updae for > 0 i (3) as + = (I + Z) Z α f( ) f( ), (5) ad he iiial sep as = Z 0 α f( 0 ). (6) By summig up he updaes i (5) ad (6) from sep 0 o ad usig he elescopic cacellaio we obai ha + = Z α f( ) ( Z Z) s. (7) s=0 We iroduce a primal-dual ierpreaio of he updae i (7) by defiig he sequece of vecors v = s=0 ( Z Z) /2 s as he accumulaio of variables dissimilariies i differe odes over ime. Noe ha if compoes of he vecor s are equal o each oher, i.e., s = = s N, he correspodig erm of he sum i he defiiio of vecor v is ull, i.e. ( Z Z) /2 s = 0. Cosiderig he defiiio of v we ca rewrie (7) as + = α f( ) + α (I Z) + α ( Z Z) /2 v. (8) Furher, based o he defiiio of sequece v = s=0 ( Z Z) /2 s we ca wrie v + as v + = v + α α ( Z Z) /2 +. (9) Cosider as he primal variable ad v as he dual variable. The, he EXTRA updae is equivale o a saddle poi mehod wih sepsize α for solvig he Lagragia L(, v) = f() + α vt ( Z Z) /2 + 2α T (I Z), (0) where he he acual Lagragia is augmeed by he quadraic erm (/2α) T (I Z). Observe ha he opimizaio problem wih he augmeed Lagragia i (0) is mi f() s.. α ( Z Z) /2 = 0. () Observig ha ull(( Z Z) /2 ) = ull( Z Z) = spa{ N I p }, he cosrai i () is equivale o = = N. Moreover, he defiiio of fucio f() i (4) shows ha he objecives of problems () ad () are also ideical. Hece, EXTRA is a saddle poi mehod ha solves () which is equivale o (). Cosiderig he eac ad liear covergece of saddle poi mehods, he covergece properies of EXTRA are jusified. III. DECENTRALIZED DOUBLE STOCHASTIC AVERAGING GRADIENT Recall he defiiios of he local fucios f ( ) ad he isaaeous local fucios f,i ( ) available a ode. To impleme EXTRA as i (3) each ode compues he full gradie of is local objecive fucio f ( ) = (/ ) i= f,i( ) which is compuaioally epesive whe he umber of isaaeous fucios is eremely large. To resolve his issue he local objecive gradies ca be subsiued by heir sochasic approimaios. The simples approach for approimaig he local objecive fucios gradie f ( ) = (/ ) i= f,i( ) is choosig a isaaeous fucio f,i ( ) radomly ad usig is gradie f,i ( ) as a ubiased esimae of he gradie f ( ) = (/ ) i= f,i( ). To be more precise, defie vecor θ = θ :... ; θ N {,..., q } {,..., q N } as a radom vecor where each compoe θ {,..., } deermies he associaed isaaeous fucio f,θ ( ) ha ode uses for gradie approimaio. To be more precise, ode i lieu of compuig he local fucio gradie f () for updaig he variable, approimaes i by f,θ ( ). However, sochasic gradies lead o a algorihm wih lower compuaio compleiy, he oise of gradie approimaio avoids eac covergece wih cosa sepsize as show for sochasic gradie desce i ceralized opimizaio. We sudy his observaio i Secio V. To overcome he oise of gradie approimaio we use he idea of ubiased sochasic averagig gradie as iroduced i 8. We iroduce he auiliary vecors φ,i R p correspodig o i-h isaaeous fucio of ode which keeps rack of he ierae for he las sep ha i-h isaaeous fucio f,i is chose a ode. To be more precise, if he ide ideifier a ime for ode is θ = i he he correspodig auiliary vecor = ad is correspodig isaaeous fucio gradie f,i (φ,i) which is sored i a memory is replaced by f,i ( ). All he oher auiliary vecors φ,j for j i ad heir correspodig isaaeous gradies remai φ,i is updaed as φ +,i uchaged, i.e. φ +,j = φ,j ad f,j (φ +,j ) = f,j(φ,j). By sorig he auiliary variables gradies f,i (φ,i), we ca defie a ubiased esimae of he local gradie f ( ) as ĝ := f,θ ( ) f,θ (φ,θ )+ i= f,i (φ,i). (2) Noice ha he sochasic approimaio ĝ is a ubiased esimae of he local gradie f ( ), i.e., E ĝ F = f ( ). The proposed sochasic averagig gradie i (2) vaishes he oise of gradie approimaio. To be more precise, as ime progresses he auiliary variables φ,i approach o a eighborhood of he opimal variable, sice hey all ge updaed over ime wih a high probabiliy. Therefore, roughly speakig we ca wrie φ,i. This propery implies ha he sochasic gradie i (2) ca be approimaed by ĝ (/ ) i= f,i(φ,i) f ( ). Therefore, he advaage of usig sochasic approimaio i (2) is he fac ha he oise of sochasic gradies is dimiishig whe he sequece is close o covergece, while for he aive approimaio f,θ ( ) he oise of sochasic approimaio ever vaishes. We iroduce Deceralized Double sochasic averagig gradie (DSA) as a sochasic versio of EXTRA ha approimaes he local gradies by heir sochasic averagig approimaios

3 Algorihm DSA algorihm a ode Require: Vecor 0 ad sored gradies f,i(φ 0,i ) wih φ0,i = 0. : for = 0,, 2,... do 2: Echage variable wih eighborig odes m N. 3: Choose θ uiformly radom from he se {,..., }. 4: Compue ad sore sochasic averagig gradie ĝ = f,θ ( ) f,θ (φ,θ ) + q f,i(φ,i q ) i= 5: Se φ + =,θ ad sore gradie f,θ (φ + i he,θ) able replacig f,θ (φ,θ ). Oher vecors of he able remai uchaged, i.e. f,j(φ +,j ) = f,j(φ,j) for j θ. 6: Updae primal variable as 7: if = 0 he 8: + = 9: else 0: + = + : ed if 2: ed for w m + αĝ. w m ŵ m α ĝ ĝ. as iroduced i (2). The DSA updae for > 0 is give by + = + w m m m= ad he iiial sep is defied as = m= w m m α ĝ ĝ, (3) w m 0 m α ĝ. 0 (4) m= To wrie he DSA updae for all he odes i oe equaio, defie he vecor ĝ := ĝ ;... ; ĝ N RNp which coais all he local sochasic averagig gradies a sep. Cosiderig his defiiio he updaes for seps > 0 i (3) ca be simplified as + = (I + Z) Z α ĝ ĝ, (5) ad he iiial updaes i (4) are equivale o = Z 0 α ĝ 0. (6) Comparig he DSA updaes i (5) ad (6) wih EXTRA seps i (5) ad (6) shows ha DSA is differe from EXTRA i usig sochasic averagig gradies ĝ i lieu of full gradies f( ). Recall ha EXTRA is a saddle poi mehod for solvig (). Therefore, DSA is a sochasic saddle poi mehod ha solves problem () where he primal variable is updaed as + = αĝ (I Z) ( Z Z) /2 v, (7) ad he dual variable v is updaed as v + = v + ( Z Z) /2 +. (8) Noice ha he iiial primal variable 0 R Np is a arbirary vecor, while accordig o he defiiio v = s=0 ( Z Z) /2 s he iiial dual vecor is se as v 0 = ( Z Z) /2 0. To impleme DSA we use he updae i (5) isead of usig he primal-dual updaes i (7) ad (8). The laer requires echage of he boh primal ad dual v variables, while for he former oly echage of he primal variables is required. The DSA algorihm is summarized i Algorihm. The updae of DSA for = 0 ad > 0 are implemeed i Seps 8 ad 0, respecively. Seps 8 ad 0 require access o he local ieraes m of he eighborig odes m N which are colleced i Sep 2. Furher, implemeaio of he DSA updae requires sochasic gradies ĝ ad ĝ which are compued i Sep 5 of ieraios ad, respecively. I Sep 3 he ide θ is chose radomly o disiguish he isaaeous fucio f,θ ha we use is gradies a pois ad φ,θ for compuig he sochasic averagig gradie i Sep 4. The able of auiliary variables gradies is updaed i Sep 5 by replacig f,θ (φ,θ ) by f,θ ( ), while he oher vecors remai uchaged. IV. CONVERGENCE ANALYSIS Our goal here is o show ha as ime progresses he sequece of ieraes approaches he opimal argume. I provig his resul for he DSA algorihm we make he followig assumpios. Assumpio. The wigh marices W ad W saisfy (a) If m ad m / N, he w m = w m = 0. (b) W = W T ad W = W T. (c) ull{ W W} = spa{} ad ull{i W} spa{}. (d) 0 W ad W W (I + W)/2. Assumpio 2. The isaaeous local fucios f,i ( ) are differeiable ad srogly cove wih parameer µ. Assumpio 3. The isaaeous local fucios gradies f,i are Lipschiz coiuous wih parameer L, f,i (a) f,i (b) L a b a, b R p. (9) The codiios imposed by Assumpio (a) o he eries of he weigh marices W ad W imply ha odes oly have access o he local ad eighborig iformaio. Furher, we assume he assiged weighs are symmeric for boh weigh marices W ad W as meioed i Assumpio (b). Codiios o he specral properies of marices W ad W i Assumpios (c) ad (d) imply ha ull{i W} = spa{} see Proposiio Assumpio 2 implies ha he local fucios f ( ) ad he global cos fucio f() = N f ( ) are srogly cove wih parameer µ. Likewise, he Lipschiz coiuiy of he local isaaeous gradies f,i ( ) eforces Lipschiz coiuiy of he local fucios gradies f ( ) ad he aggregae fucio gradies f(). Defie 0 < γ ad Γ < as he smalles ad larges eigevalues of Z, respecively. Likewise, defie γ as he smalles o-zero eigevalue of he mari Z Z ad Γ as he larges eigevalue of Z Z. Furher, defie vecors u, u R 2Np ad mari G R 2Np 2Np as u :=, u :=, G = v v Z 0. (20) 0 I The vecor u R 2Np cocaeaes he opimal primal ad dual variables ad he vecor u R 2Np coais primal ad dual ieraes a sep. Furher, G R 2Np 2Np is a block diagoal posiive defiie mari. We sudy he covergece properies of he weighed orm u u 2 G which is equivale o (u u ) T G(u u ). Our goal is o show ha he sequece u u 2 G coverges liearly o ull. To do his we show liear covergece of a Lyapuov fucio of he sequece u u 2 G.

4 The Lyapuov fucio is defied as u u 2 G + cp where p := i= f,i (φ,i) f ( ) i= f,i ( ) T (φ,i ), (2) ad c > 0 is a posiive cosa. Noice ha based o he srog coveiy of he local isaaeous fucios f,i, each erm f,i (φ,i) f,i ( ) f,i ( ) T (φ,i ) is posiive ad as a resul he sequece p defied i (2) is always posiive. To prove liear covergece of he sequece u u 2 G + cp we firs show a upper boud for he epeced error E u + u 2 G F i erms of u u 2 G ad some parameers ha capure he opimaliy gap. Lemma. Cosider he DSA algorihm as defied i (2)-(8). Furher, recall he defiiios of p i (2) ad u, u, ad G i (20). If Assumpios -3 hold rue, he for ay posiive cosas η, ρ > 0 we ca wrie E u + u 2 G F u u 2 G (22) E + 2 Z α(η+ρ)i F 2E + F 2I+Z 2 Z E v + v 2 F + 4αL ρ p αc 0 2, where C 0 = (2µ 2 /L) (L 2 /η) (2(L 2 µ 2 ))/ρ. Likewise, we provide a upper boud for he oher par of he Lyapuov fucio a ime + which is p + i erms of p ad some parameers ha capure opimaliy gap. This boud is sudied i he followig lemma. Lemma 2. Cosider he DSA algorihm as defied i (2)-(8) ad he defiiio of p i (2). Furher, defie q mi ad q ma as he smalles ad larges values for he size of isaaeous fucios a a ode, respecively. If Assumpios -3 hold rue, he for all > 0 E p + F ( ) p + q ma L 2q mi 2. (23) Combiig he resuls i Lemmaa ad 2 we ca show ha he epeced Lyapuov fucio E u + u 2 G + c p+ F is sricly smaller ha is previous value u u 2 G + c p. Theorem. Cosider he DSA algorihm as defied i (2)- (8). Furher, recall he defiiios of p i (2) ad u, u, ad G i (20). Moreover, cosider he resuls i (22) ad (23). If Assumpios -3 hold rue ad he sepsize α ad he parameer c are chose properly he here eis 0 < δ < such ha E u + u 2 G + c p + F ( δ) ( u u 2 G + c p ). (24) The codiios o α ad c, ad he eplici epressio of δ are provided i 9. The iequaliy i (24) shows ha he epeced value of he sequece u + u 2 G + c p+ give he observaios uil sep is sricly smaller ha he previous ierae u u 2 G +cp. By akig he epeced value wih respec o he iiial field E. F 0 = E. ad applyig he implied iequaliy recursively we obai ha E u u 2 G + c p ( δ) ( u 0 u 2 G + c p 0). (25) Accordig o (25), he sequece u u 2 G + c p coverges liearly o ull i epecaio. Noice ha he orm u u 2 G is equal o 2 Z + v v 2. Hece, he iequaliy 2 Z u u 2 G holds. Moreover, he sequece p is always o-egaive. Cosiderig hese wo observaios he iequaliy 2 Z u u 2 G + c p holds rue. Cosiderig his iequaliy ad he epressio i (25), ad observig ha he erm 2 Z is lower bouded by γ 2, we ca wrie he followig corollary. Corollary. Cosider he DSA algorihm as defied i (2)-(8). Recall he defiiios of p i (2) ad u, u, ad G i (20). Furher, recall γ as he smalles eigevalue of Z. If Assumpios -3 hold rue, he here eis a cosa 0 < δ < such ha E 2 ( δ) ( u 0 u 2 G + c p0). γ (26) Corollary saes ha E 2 liearly coverges o ull. Noe ha he sequece E 2 is o ecessarily decreasig as he sequece E u u 2 G + c p is. V. NUMERICAL ANALYSIS We umerically sudy he performace of he DSA algorihm for solvig a logisic regressio problem. Cosider q = N raiig pois where each ode has access o of hem. The raiig pois a ode are deoed by s i R p for i =,..., wih he associaed labels l i {, }. The goal is o solve he logisic regressio problem := argmi R p i= log + ep( l i s T i), (27) where he regularizaio erm (/2) 2 is added o avoid overfiig he raiig model. The problem i (27) ca be wrie i he form of () by defiig he local objecive fucios f as f () = 2N 2 + i= ad he isaaeous local fucios f,i as log + ep( l i s T i), (28) f,i () = 2N 2 + log ( + ep ( l i s T i )), (29) for all i =,...,. I our eperimes we use a syheic daase where compoes of he feaure vecors s i wih label l i = are geeraed from a ormal disribuio wih mea µ ad sadard deviaio σ +, while he disribuio of he sample pois wih label l i = is ormal wih mea µ ad sadard deviaio σ. The edges bewee odes are geeraed radomly wih probabiliy p c. The weigh mari W is geeraed usig he Laplacia mari L of he ework as W = I L/τ, where τ > (/2) ma (L). The covergece error is defied ad compued as e = 2. We se he oal umber of sample pois q = 500, feaure vecors dimesio p = 2, regularizaio parameer = 0 4, probabiliy of eisece of a edge p c = 0.3, ad τ = (2/3) ma (L). To make he daase o liearly separable we se he mea value as µ = 2 ad he sadard deviaios as σ + = σ = 2. We provide a compariso of DSA wih respec o DGD, EX- TRA, sochasic EXTRA, ad deceralized SAGA. The sochasic EXTRA is a sochasic versio of EXTRA ha uses aive sochasic gradies isead of gradies. DSA is differe form sochasic EXTRA sice i uses sochasic averagig gradies.

5 Radom e. pc = 0.2 Radom e. pc = 0.3 Complee e. Cycle Lie Sar error e EXTRA α = DSA α = DGD α = 0 3 D-SAG α = 0 3 DGD α = 0 2 D-SAG α = 0 2 so-extra α = 0 3 so-extra α = 0 2 error e Number of ieraios Number of ieraios Fig.. Covergece pahs of DSA, EXTRA, DGD, Sochasic EXTRA, ad Deceralized SAGA wih cosa sepsizes. Relaive disace o opimaliy e = 2 is show wih respec o he umber ieraios. DSA ad EXTRA coverge liearly o he opimum, while DGD, Sochasic EXTRA, ad Deceralized SAGA coverge o a eighborhood of he opimal soluio. Smaller choice of sepsize leads o more accurae covergece for hese algorihms. The deceralized SAGA mehod is a sochasic versio of DGD algorihm ha uses sochasic averagig gradies isead of gradies which is a aive approach for developig a deceralized versio of he SAGA algorihm. I our eperimes we use W = (I + W)/2 for EXTRA, sochasic EXTRA, ad DSA. Fig. illusraes he covergece pahs of DSA, EXTRA, DGD, Sochasic EXTRA, ad Deceralized SAGA wih cosa sep sizes for N = 20 odes. For EXTRA ad DSA differe sepsize are chose ad he bes performace for EXTRA ad DSA are achieved by α = ad α = 5 0 3, respecively. As show i Fig., DSA is he oly sochasic algorihm ha achieves liear covergece. Deceralized SAGA afer couple of ieraios achieves he performace of DGD ad hey boh ca o achieve eac covergece. By choosig smaller sepsize α = 0 3 hey reach more accurae covergece relaive o sepsize α = 0 2, however, he speed of covergece is slower for he smaller sepsize. Sochasic EXTRA also suffers from ieac covergece, bu for a differe reaso. DGD ad deceralized SAGA have ieac covergece sice hey solve a pealy versio of (), while sochasic EXTRA ca o reach he opimal soluio sice he oise of sochasic gradie is o vaishig. DSA resolves boh issues by combiig he idea of sochasic averagig gradies o corol he oise of sochasic gradies ad usig he double deceralized desce idea o solve he correc opimizaio problem. The covergece rae of EXTRA is faser ha he oe for DSA i erms of umber of ieraios, however, he compleiy of EXTRA is higher ha DSA. Hece, we also compare performaces of hese algorihms i erms of umber of processed feaure vecors. For isace, DSA requires 400 ieraios or equivalely 400 feaure vecors o achieve error e = 0 7, while o achieve he same accuracy EXTRA requires 60 ieraios which is equivale o processig = 440 feaure vecors. The differece ca be more sigifica by icreasig he umber of isaaeous fucios. We also sudy he performace of DSA i differe ework opologies. We keep he parameers i Fig. ecep we chage he size of ework o N = 00 which implies each ode has q i = 5 sample pois. The liear covergece of DSA for radom eworks, complee graph, cycle, lie ad sar are show i Fig. 2. As we epec for he opologies ha he graph is more coeced ad he diameer is smaller DSA coverges faser. Fig. 2. Covergece pahs of DSA for differe ework opologies. Relaive disace o opimaliy e = 2 is show wih respec o he umber ieraios. DSA coverges faser as he ework coeciviy icreases. REFERENCES R. Bekkerma, M. Bileko, ad J. Lagford, Scalig up machie learig: Parallel ad disribued approaches. Cambridge Uiversiy Press, K. Tsiaos, S. Lawlor, ad M. Rabba, Cosesus-based disribued opimizaio: Pracical issues ad applicaios i large-scale machie learig, I: Proceedigs of Allero Coferece o Commuicaio, Corol, ad Compuig, Y. Cao, W. Yu, W. Re, ad G. Che, A overview of rece progress i he sudy of disribued muli-age coordiaio, IEEE Trasacios o Idusrial Iformaics, vol. 9, pp , C. G. Lopes ad A. H. Sayed, Diffusio leas-mea squares over adapive eworks: Formulaio ad performace aalysis, IEEE Trasacios o Sigal Processig, vol. 56, o. 7, pp , July I. Schizas, A. Ribeiro, ad G. Giaakis, Cosesus i ad hoc wss wih oisy liks - par i: Disribued esimaio of deermiisic sigals, IEEE Trasacios o Sigal Processig, vol. 56, pp , U. A. Kha, S. Kar, ad J. M. Moura, Dilad: A algorihm for disribued sesor localizaio wih oisy disace measuremes, IEEE Trasacios o Sigal Processig, vol. 58, o. 3, pp , M. Rabba ad R. Nowak, Disribued opimizaio i sesor eworks, proceedigs of he 3rd ieraioal symposium o Iformaio processig i sesor eworks, pp , ACM, A. Nedic ad A. Ozdaglar, Disribued subgradie mehods for muliage opimizaio, IEEE Trasacios o Auomaic Corol, vol. 54, pp. 48 6, D. Jakoveic, J. Xavier, ad J. Moura, Fas disribued gradie mehods, IEEE Trasacios o Auomaic Corol, vol. 59, pp. 3 46, K. Yua, Q. Lig, ad W. Yi, O he covergece of deceralized gradie desce, arxiv prepri arxiv, , 203. A. Mokhari, Q. Lig, ad A. Ribeiro, Nework ewo, i Proc. Asilomar Cof. o Sigals Sysems Compuers, vol. (o appear). Pacific Grove CA, November , available a hp://ariv.org/pdf/ pdf. 2 W. Shi, Q. Lig, G. Wu, ad W. Yi, Era: A eac firs-order algorihm for deceralized cosesus opimizaio, arxiv prepri arxiv, J. N. Tsisiklis, D. P. Bersekas, M. Ahas e al., Disribued asychroous deermiisic ad sochasic gradie opimizaio algorihms, IEEE rasacios o auomaic corol, vol. 3, o. 9, pp , J. Duchi, A. Agarwal, ad M. Waiwrigh, Dual averagig for disribued opimizaio: Covergece aalysis ad ework scalig, IEEE Trasacios o Auomaic Corol, vol. 57, pp , S. S. Ram, A. Nedic, ad V. V. Veeravalli, Icremeal sochasic subgradie algorihms for cove opimizaio, SIAM Joural o Opimizaio, vol. 20, o. 2, pp , S. S. Ram, A. Nedić, ad V. V. Veeravalli, Disribued sochasic subgradie projecio algorihms for cove opimizaio, Joural of opimizaio heory ad applicaios, vol. 47, o. 3, pp , M. Schmid, N. L. Rou, ad F. Bach, Miimizig fiie sums wih he sochasic average gradie, arxiv prepri arxiv: , A. Defazio, F. Bach, ad S. Lacose-Julie, Saga: A fas icremeal gradie mehod wih suppor for o-srogly cove composie objecives, i Advaces i Neural Iformaio Processig Sysems, 204, pp A. Mokhari ad A. Ribeiro, Deceralized double sochasic averagig gradie, 205, Olie:hp:// aryam/wiki/dsa.pdf.

Supplement for SADAGRAD: Strongly Adaptive Stochastic Gradient Methods"

Supplement for SADAGRAD: Strongly Adaptive Stochastic Gradient Methods Suppleme for SADAGRAD: Srogly Adapive Sochasic Gradie Mehods" Zaiyi Che * 1 Yi Xu * Ehog Che 1 iabao Yag 1. Proof of Proposiio 1 Proposiio 1. Le ɛ > 0 be fixed, H 0 γi, γ g, EF (w 1 ) F (w ) ɛ 0 ad ieraio

More information

1 Notes on Little s Law (l = λw)

1 Notes on Little s Law (l = λw) Copyrigh c 26 by Karl Sigma Noes o Lile s Law (l λw) We cosider here a famous ad very useful law i queueig heory called Lile s Law, also kow as l λw, which assers ha he ime average umber of cusomers i

More information

Comparison between Fourier and Corrected Fourier Series Methods

Comparison between Fourier and Corrected Fourier Series Methods Malaysia Joural of Mahemaical Scieces 7(): 73-8 (13) MALAYSIAN JOURNAL OF MATHEMATICAL SCIENCES Joural homepage: hp://eispem.upm.edu.my/oural Compariso bewee Fourier ad Correced Fourier Series Mehods 1

More information

STK4080/9080 Survival and event history analysis

STK4080/9080 Survival and event history analysis STK48/98 Survival ad eve hisory aalysis Marigales i discree ime Cosider a sochasic process The process M is a marigale if Lecure 3: Marigales ad oher sochasic processes i discree ime (recap) where (formally

More information

λiv Av = 0 or ( λi Av ) = 0. In order for a vector v to be an eigenvector, it must be in the kernel of λi

λiv Av = 0 or ( λi Av ) = 0. In order for a vector v to be an eigenvector, it must be in the kernel of λi Liear lgebra Lecure #9 Noes This week s lecure focuses o wha migh be called he srucural aalysis of liear rasformaios Wha are he irisic properies of a liear rasformaio? re here ay fixed direcios? The discussio

More information

ODEs II, Supplement to Lectures 6 & 7: The Jordan Normal Form: Solving Autonomous, Homogeneous Linear Systems. April 2, 2003

ODEs II, Supplement to Lectures 6 & 7: The Jordan Normal Form: Solving Autonomous, Homogeneous Linear Systems. April 2, 2003 ODEs II, Suppleme o Lecures 6 & 7: The Jorda Normal Form: Solvig Auoomous, Homogeeous Liear Sysems April 2, 23 I his oe, we describe he Jorda ormal form of a marix ad use i o solve a geeral homogeeous

More information

Big O Notation for Time Complexity of Algorithms

Big O Notation for Time Complexity of Algorithms BRONX COMMUNITY COLLEGE of he Ciy Uiversiy of New York DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE CSI 33 Secio E01 Hadou 1 Fall 2014 Sepember 3, 2014 Big O Noaio for Time Complexiy of Algorihms Time

More information

Additional Tables of Simulation Results

Additional Tables of Simulation Results Saisica Siica: Suppleme REGULARIZING LASSO: A CONSISTENT VARIABLE SELECTION METHOD Quefeg Li ad Ju Shao Uiversiy of Wiscosi, Madiso, Eas Chia Normal Uiversiy ad Uiversiy of Wiscosi, Madiso Supplemeary

More information

Lecture 15 First Properties of the Brownian Motion

Lecture 15 First Properties of the Brownian Motion Lecure 15: Firs Properies 1 of 8 Course: Theory of Probabiliy II Term: Sprig 2015 Isrucor: Gorda Zikovic Lecure 15 Firs Properies of he Browia Moio This lecure deals wih some of he more immediae properies

More information

Ideal Amplifier/Attenuator. Memoryless. where k is some real constant. Integrator. System with memory

Ideal Amplifier/Attenuator. Memoryless. where k is some real constant. Integrator. System with memory Liear Time-Ivaria Sysems (LTI Sysems) Oulie Basic Sysem Properies Memoryless ad sysems wih memory (saic or dyamic) Causal ad o-causal sysems (Causaliy) Liear ad o-liear sysems (Lieariy) Sable ad o-sable

More information

Available online at J. Math. Comput. Sci. 4 (2014), No. 4, ISSN:

Available online at   J. Math. Comput. Sci. 4 (2014), No. 4, ISSN: Available olie a hp://sci.org J. Mah. Compu. Sci. 4 (2014), No. 4, 716-727 ISSN: 1927-5307 ON ITERATIVE TECHNIQUES FOR NUMERICAL SOLUTIONS OF LINEAR AND NONLINEAR DIFFERENTIAL EQUATIONS S.O. EDEKI *, A.A.

More information

FIXED FUZZY POINT THEOREMS IN FUZZY METRIC SPACE

FIXED FUZZY POINT THEOREMS IN FUZZY METRIC SPACE Mohia & Samaa, Vol. 1, No. II, December, 016, pp 34-49. ORIGINAL RESEARCH ARTICLE OPEN ACCESS FIED FUZZY POINT THEOREMS IN FUZZY METRIC SPACE 1 Mohia S. *, Samaa T. K. 1 Deparme of Mahemaics, Sudhir Memorial

More information

Review Answers for E&CE 700T02

Review Answers for E&CE 700T02 Review Aswers for E&CE 700T0 . Deermie he curre soluio, all possible direcios, ad sepsizes wheher improvig or o for he simple able below: 4 b ma c 0 0 0-4 6 0 - B N B N ^0 0 0 curre sol =, = Ch for - -

More information

10.3 Autocorrelation Function of Ergodic RP 10.4 Power Spectral Density of Ergodic RP 10.5 Normal RP (Gaussian RP)

10.3 Autocorrelation Function of Ergodic RP 10.4 Power Spectral Density of Ergodic RP 10.5 Normal RP (Gaussian RP) ENGG450 Probabiliy ad Saisics for Egieers Iroducio 3 Probabiliy 4 Probabiliy disribuios 5 Probabiliy Desiies Orgaizaio ad descripio of daa 6 Samplig disribuios 7 Ifereces cocerig a mea 8 Comparig wo reames

More information

CS623: Introduction to Computing with Neural Nets (lecture-10) Pushpak Bhattacharyya Computer Science and Engineering Department IIT Bombay

CS623: Introduction to Computing with Neural Nets (lecture-10) Pushpak Bhattacharyya Computer Science and Engineering Department IIT Bombay CS6: Iroducio o Compuig ih Neural Nes lecure- Pushpak Bhaacharyya Compuer Sciece ad Egieerig Deparme IIT Bombay Tilig Algorihm repea A kid of divide ad coquer sraegy Give he classes i he daa, ru he percepro

More information

Four equations describe the dynamic solution to RBC model. Consumption-leisure efficiency condition. Consumption-investment efficiency condition

Four equations describe the dynamic solution to RBC model. Consumption-leisure efficiency condition. Consumption-investment efficiency condition LINEAR APPROXIMATION OF THE BASELINE RBC MODEL FEBRUARY, 202 Iroducio For f(, y, z ), mulivariable Taylor liear epasio aroud (, yz, ) f (, y, z) f(, y, z) + f (, y, z)( ) + f (, y, z)( y y) + f (, y, z)(

More information

BEST LINEAR FORECASTS VS. BEST POSSIBLE FORECASTS

BEST LINEAR FORECASTS VS. BEST POSSIBLE FORECASTS BEST LINEAR FORECASTS VS. BEST POSSIBLE FORECASTS Opimal ear Forecasig Alhough we have o meioed hem explicily so far i he course, here are geeral saisical priciples for derivig he bes liear forecas, ad

More information

NEWTON METHOD FOR DETERMINING THE OPTIMAL REPLENISHMENT POLICY FOR EPQ MODEL WITH PRESENT VALUE

NEWTON METHOD FOR DETERMINING THE OPTIMAL REPLENISHMENT POLICY FOR EPQ MODEL WITH PRESENT VALUE Yugoslav Joural of Operaios Research 8 (2008, Number, 53-6 DOI: 02298/YUJOR080053W NEWTON METHOD FOR DETERMINING THE OPTIMAL REPLENISHMENT POLICY FOR EPQ MODEL WITH PRESENT VALUE Jeff Kuo-Jug WU, Hsui-Li

More information

Lecture 9: Polynomial Approximations

Lecture 9: Polynomial Approximations CS 70: Complexiy Theory /6/009 Lecure 9: Polyomial Approximaios Isrucor: Dieer va Melkebeek Scribe: Phil Rydzewski & Piramaayagam Arumuga Naiar Las ime, we proved ha o cosa deph circui ca evaluae he pariy

More information

The Solution of the One Species Lotka-Volterra Equation Using Variational Iteration Method ABSTRACT INTRODUCTION

The Solution of the One Species Lotka-Volterra Equation Using Variational Iteration Method ABSTRACT INTRODUCTION Malaysia Joural of Mahemaical Scieces 2(2): 55-6 (28) The Soluio of he Oe Species Loka-Volerra Equaio Usig Variaioal Ieraio Mehod B. Baiha, M.S.M. Noorai, I. Hashim School of Mahemaical Scieces, Uiversii

More information

Online Supplement to Reactive Tabu Search in a Team-Learning Problem

Online Supplement to Reactive Tabu Search in a Team-Learning Problem Olie Suppleme o Reacive abu Search i a eam-learig Problem Yueli She School of Ieraioal Busiess Admiisraio, Shaghai Uiversiy of Fiace ad Ecoomics, Shaghai 00433, People s Republic of Chia, she.yueli@mail.shufe.edu.c

More information

State and Parameter Estimation of The Lorenz System In Existence of Colored Noise

State and Parameter Estimation of The Lorenz System In Existence of Colored Noise Sae ad Parameer Esimaio of he Lorez Sysem I Eisece of Colored Noise Mozhga Mombeii a Hamid Khaloozadeh b a Elecrical Corol ad Sysem Egieerig Researcher of Isiue for Research i Fudameal Scieces (IPM ehra

More information

Outline. simplest HMM (1) simple HMMs? simplest HMM (2) Parameter estimation for discrete hidden Markov models

Outline. simplest HMM (1) simple HMMs? simplest HMM (2) Parameter estimation for discrete hidden Markov models Oulie Parameer esimaio for discree idde Markov models Juko Murakami () ad Tomas Taylor (2). Vicoria Uiversiy of Welligo 2. Arizoa Sae Uiversiy Descripio of simple idde Markov models Maximum likeliood esimae

More information

Mean Square Convergent Finite Difference Scheme for Stochastic Parabolic PDEs

Mean Square Convergent Finite Difference Scheme for Stochastic Parabolic PDEs America Joural of Compuaioal Mahemaics, 04, 4, 80-88 Published Olie Sepember 04 i SciRes. hp://www.scirp.org/joural/ajcm hp://dx.doi.org/0.436/ajcm.04.4404 Mea Square Coverge Fiie Differece Scheme for

More information

SUMMATION OF INFINITE SERIES REVISITED

SUMMATION OF INFINITE SERIES REVISITED SUMMATION OF INFINITE SERIES REVISITED I several aricles over he las decade o his web page we have show how o sum cerai iiie series icludig he geomeric series. We wa here o eed his discussio o he geeral

More information

Order Determination for Multivariate Autoregressive Processes Using Resampling Methods

Order Determination for Multivariate Autoregressive Processes Using Resampling Methods joural of mulivariae aalysis 57, 175190 (1996) aricle o. 0028 Order Deermiaio for Mulivariae Auoregressive Processes Usig Resamplig Mehods Chaghua Che ad Richard A. Davis* Colorado Sae Uiversiy ad Peer

More information

A TAUBERIAN THEOREM FOR THE WEIGHTED MEAN METHOD OF SUMMABILITY

A TAUBERIAN THEOREM FOR THE WEIGHTED MEAN METHOD OF SUMMABILITY U.P.B. Sci. Bull., Series A, Vol. 78, Iss. 2, 206 ISSN 223-7027 A TAUBERIAN THEOREM FOR THE WEIGHTED MEAN METHOD OF SUMMABILITY İbrahim Çaak I his paper we obai a Tauberia codiio i erms of he weighed classical

More information

SIGNAL estimation has been a fundamental problem in a

SIGNAL estimation has been a fundamental problem in a 90 IEEE TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING OVER NETWORKS, VOL., NO. 3, SEPTEMBER 06 A Olie Parallel Algorihm for Recursive Esimaio of Sparse Sigals Yag Yag, Member, IEEE, Marius Pesaveo,

More information

The analysis of the method on the one variable function s limit Ke Wu

The analysis of the method on the one variable function s limit Ke Wu Ieraioal Coferece o Advaces i Mechaical Egieerig ad Idusrial Iformaics (AMEII 5) The aalysis of he mehod o he oe variable fucio s i Ke Wu Deparme of Mahemaics ad Saisics Zaozhuag Uiversiy Zaozhuag 776

More information

Some Properties of Semi-E-Convex Function and Semi-E-Convex Programming*

Some Properties of Semi-E-Convex Function and Semi-E-Convex Programming* The Eighh Ieraioal Symposium o Operaios esearch ad Is Applicaios (ISOA 9) Zhagjiajie Chia Sepember 2 22 29 Copyrigh 29 OSC & APOC pp 33 39 Some Properies of Semi-E-Covex Fucio ad Semi-E-Covex Programmig*

More information

Extremal graph theory II: K t and K t,t

Extremal graph theory II: K t and K t,t Exremal graph heory II: K ad K, Lecure Graph Theory 06 EPFL Frak de Zeeuw I his lecure, we geeralize he wo mai heorems from he las lecure, from riagles K 3 o complee graphs K, ad from squares K, o complee

More information

Notes 03 largely plagiarized by %khc

Notes 03 largely plagiarized by %khc 1 1 Discree-Time Covoluio Noes 03 largely plagiarized by %khc Le s begi our discussio of covoluio i discree-ime, sice life is somewha easier i ha domai. We sar wih a sigal x[] ha will be he ipu io our

More information

Math 6710, Fall 2016 Final Exam Solutions

Math 6710, Fall 2016 Final Exam Solutions Mah 67, Fall 6 Fial Exam Soluios. Firs, a sude poied ou a suble hig: if P (X i p >, he X + + X (X + + X / ( evaluaes o / wih probabiliy p >. This is roublesome because a radom variable is supposed o be

More information

F D D D D F. smoothed value of the data including Y t the most recent data.

F D D D D F. smoothed value of the data including Y t the most recent data. Module 2 Forecasig 1. Wha is forecasig? Forecasig is defied as esimaig he fuure value ha a parameer will ake. Mos scieific forecasig mehods forecas he fuure value usig pas daa. I Operaios Maageme forecasig

More information

Actuarial Society of India

Actuarial Society of India Acuarial Sociey of Idia EXAMINAIONS Jue 5 C4 (3) Models oal Marks - 5 Idicaive Soluio Q. (i) a) Le U deoe he process described by 3 ad V deoe he process described by 4. he 5 e 5 PU [ ] PV [ ] ( e ).538!

More information

Review Exercises for Chapter 9

Review Exercises for Chapter 9 0_090R.qd //0 : PM Page 88 88 CHAPTER 9 Ifiie Series I Eercises ad, wrie a epressio for he h erm of he sequece..,., 5, 0,,,, 0,... 7,... I Eercises, mach he sequece wih is graph. [The graphs are labeled

More information

A Complex Neural Network Algorithm for Computing the Largest Real Part Eigenvalue and the corresponding Eigenvector of a Real Matrix

A Complex Neural Network Algorithm for Computing the Largest Real Part Eigenvalue and the corresponding Eigenvector of a Real Matrix 4h Ieraioal Coferece o Sesors, Mecharoics ad Auomaio (ICSMA 06) A Complex Neural Newor Algorihm for Compuig he Larges eal Par Eigevalue ad he correspodig Eigevecor of a eal Marix HANG AN, a, XUESONG LIANG,

More information

1. Solve by the method of undetermined coefficients and by the method of variation of parameters. (4)

1. Solve by the method of undetermined coefficients and by the method of variation of parameters. (4) 7 Differeial equaios Review Solve by he mehod of udeermied coefficies ad by he mehod of variaio of parameers (4) y y = si Soluio; we firs solve he homogeeous equaio (4) y y = 4 The correspodig characerisic

More information

11. Adaptive Control in the Presence of Bounded Disturbances Consider MIMO systems in the form,

11. Adaptive Control in the Presence of Bounded Disturbances Consider MIMO systems in the form, Lecure 6. Adapive Corol i he Presece of Bouded Disurbaces Cosider MIMO sysems i he form, x Aref xbu x Bref ycmd (.) y Cref x operaig i he presece of a bouded ime-depede disurbace R. All he assumpios ad

More information

Economics 8723 Macroeconomic Theory Problem Set 2 Professor Sanjay Chugh Spring 2017

Economics 8723 Macroeconomic Theory Problem Set 2 Professor Sanjay Chugh Spring 2017 Deparme of Ecoomics The Ohio Sae Uiversiy Ecoomics 8723 Macroecoomic Theory Problem Se 2 Professor Sajay Chugh Sprig 207 Labor Icome Taxes, Nash-Bargaied Wages, ad Proporioally-Bargaied Wages. I a ecoomy

More information

Section 8 Convolution and Deconvolution

Section 8 Convolution and Deconvolution APPLICATIONS IN SIGNAL PROCESSING Secio 8 Covoluio ad Decovoluio This docume illusraes several echiques for carryig ou covoluio ad decovoluio i Mahcad. There are several operaors available for hese fucios:

More information

Solution. 1 Solutions of Homework 6. Sangchul Lee. April 28, Problem 1.1 [Dur10, Exercise ]

Solution. 1 Solutions of Homework 6. Sangchul Lee. April 28, Problem 1.1 [Dur10, Exercise ] Soluio Sagchul Lee April 28, 28 Soluios of Homework 6 Problem. [Dur, Exercise 2.3.2] Le A be a sequece of idepede eves wih PA < for all. Show ha P A = implies PA i.o. =. Proof. Noice ha = P A c = P A c

More information

If boundary values are necessary, they are called mixed initial-boundary value problems. Again, the simplest prototypes of these IV problems are:

If boundary values are necessary, they are called mixed initial-boundary value problems. Again, the simplest prototypes of these IV problems are: 3. Iiial value problems: umerical soluio Fiie differeces - Trucaio errors, cosisecy, sabiliy ad covergece Crieria for compuaioal sabiliy Explici ad implici ime schemes Table of ime schemes Hyperbolic ad

More information

Asymptotic statistics for multilayer perceptron with ReLu hidden units

Asymptotic statistics for multilayer perceptron with ReLu hidden units ESANN 8 proceedigs, Europea Symposium o Arificial Neural Neworks, Compuaioal Ielligece ad Machie Learig. Bruges (Belgium), 5-7 April 8, i6doc.com publ., ISBN 978-8758747-6. Available from hp://www.i6doc.com/e/.

More information

COS 522: Complexity Theory : Boaz Barak Handout 10: Parallel Repetition Lemma

COS 522: Complexity Theory : Boaz Barak Handout 10: Parallel Repetition Lemma COS 522: Complexiy Theory : Boaz Barak Hadou 0: Parallel Repeiio Lemma Readig: () A Parallel Repeiio Theorem / Ra Raz (available o his websie) (2) Parallel Repeiio: Simplificaios ad he No-Sigallig Case

More information

Calculus BC 2015 Scoring Guidelines

Calculus BC 2015 Scoring Guidelines AP Calculus BC 5 Scorig Guidelies 5 The College Board. College Board, Advaced Placeme Program, AP, AP Ceral, ad he acor logo are regisered rademarks of he College Board. AP Ceral is he official olie home

More information

A Generalized Cost Malmquist Index to the Productivities of Units with Negative Data in DEA

A Generalized Cost Malmquist Index to the Productivities of Units with Negative Data in DEA Proceedigs of he 202 Ieraioal Coferece o Idusrial Egieerig ad Operaios Maageme Isabul, urey, July 3 6, 202 A eeralized Cos Malmquis Ide o he Produciviies of Uis wih Negaive Daa i DEA Shabam Razavya Deparme

More information

A note on deviation inequalities on {0, 1} n. by Julio Bernués*

A note on deviation inequalities on {0, 1} n. by Julio Bernués* A oe o deviaio iequaliies o {0, 1}. by Julio Berués* Deparameo de Maemáicas. Faculad de Ciecias Uiversidad de Zaragoza 50009-Zaragoza (Spai) I. Iroducio. Le f: (Ω, Σ, ) IR be a radom variable. Roughly

More information

Calculus Limits. Limit of a function.. 1. One-Sided Limits...1. Infinite limits 2. Vertical Asymptotes...3. Calculating Limits Using the Limit Laws.

Calculus Limits. Limit of a function.. 1. One-Sided Limits...1. Infinite limits 2. Vertical Asymptotes...3. Calculating Limits Using the Limit Laws. Limi of a fucio.. Oe-Sided..... Ifiie limis Verical Asympoes... Calculaig Usig he Limi Laws.5 The Squeeze Theorem.6 The Precise Defiiio of a Limi......7 Coiuiy.8 Iermediae Value Theorem..9 Refereces..

More information

ANALYSIS OF THE CHAOS DYNAMICS IN (X n,x n+1) PLANE

ANALYSIS OF THE CHAOS DYNAMICS IN (X n,x n+1) PLANE ANALYSIS OF THE CHAOS DYNAMICS IN (X,X ) PLANE Soegiao Soelisioo, The Houw Liog Badug Isiue of Techolog (ITB) Idoesia soegiao@sude.fi.ib.ac.id Absrac I he las decade, sudies of chaoic ssem are more ofe

More information

An Online Parallel and Distributed Algorithm for Recursive Estimation of Sparse Signals

An Online Parallel and Distributed Algorithm for Recursive Estimation of Sparse Signals A Olie Parallel ad Disribued Algorihm for Recursive simaio of Sparse Sigals Yag Yag, Megyi Zhag, Marius Pesaveo, ad Daiel P. Palomar Absrac I his paper, we cosider a recursive esimaio problem for liear

More information

Approximating Solutions for Ginzburg Landau Equation by HPM and ADM

Approximating Solutions for Ginzburg Landau Equation by HPM and ADM Available a hp://pvamu.edu/aam Appl. Appl. Mah. ISSN: 193-9466 Vol. 5, No. Issue (December 1), pp. 575 584 (Previously, Vol. 5, Issue 1, pp. 167 1681) Applicaios ad Applied Mahemaics: A Ieraioal Joural

More information

INTEGER INTERVAL VALUE OF NEWTON DIVIDED DIFFERENCE AND FORWARD AND BACKWARD INTERPOLATION FORMULA

INTEGER INTERVAL VALUE OF NEWTON DIVIDED DIFFERENCE AND FORWARD AND BACKWARD INTERPOLATION FORMULA Volume 8 No. 8, 45-54 ISSN: 34-3395 (o-lie versio) url: hp://www.ijpam.eu ijpam.eu INTEGER INTERVAL VALUE OF NEWTON DIVIDED DIFFERENCE AND FORWARD AND BACKWARD INTERPOLATION FORMULA A.Arul dass M.Dhaapal

More information

Mathematical Statistics. 1 Introduction to the materials to be covered in this course

Mathematical Statistics. 1 Introduction to the materials to be covered in this course Mahemaical Saisics Iroducio o he maerials o be covered i his course. Uivariae & Mulivariae r.v s 2. Borl-Caelli Lemma Large Deviaios. e.g. X,, X are iid r.v s, P ( X + + X where I(A) is a umber depedig

More information

Lecture 8 April 18, 2018

Lecture 8 April 18, 2018 Sas 300C: Theory of Saisics Sprig 2018 Lecure 8 April 18, 2018 Prof Emmauel Cades Scribe: Emmauel Cades Oulie Ageda: Muliple Tesig Problems 1 Empirical Process Viewpoi of BHq 2 Empirical Process Viewpoi

More information

Enhanced Online Subspace Estimation via Adaptive Sensing

Enhanced Online Subspace Estimation via Adaptive Sensing Ehaced Olie Subspace Esimaio via Adapive Sesig Greg Ogie, David Hog, Dejiao Zhag, Laura Balzao Deparme of Elecrical Egieerig ad Compuer Sciece Uiversiy of Michiga A Arbor, MI 488 e-mail: {gogie,dahog,dejiao,girasole}@umich.edu

More information

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 4 9/16/2013. Applications of the large deviation technique

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 4 9/16/2013. Applications of the large deviation technique MASSACHUSETTS ISTITUTE OF TECHOLOGY 6.265/5.070J Fall 203 Lecure 4 9/6/203 Applicaios of he large deviaio echique Coe.. Isurace problem 2. Queueig problem 3. Buffer overflow probabiliy Safey capial for

More information

Procedia - Social and Behavioral Sciences 230 ( 2016 ) Joint Probability Distribution and the Minimum of a Set of Normalized Random Variables

Procedia - Social and Behavioral Sciences 230 ( 2016 ) Joint Probability Distribution and the Minimum of a Set of Normalized Random Variables Available olie a wwwsciecedireccom ScieceDirec Procedia - Social ad Behavioral Scieces 30 ( 016 ) 35 39 3 rd Ieraioal Coferece o New Challeges i Maageme ad Orgaizaio: Orgaizaio ad Leadership, May 016,

More information

David Randall. ( )e ikx. k = u x,t. u( x,t)e ikx dx L. x L /2. Recall that the proof of (1) and (2) involves use of the orthogonality condition.

David Randall. ( )e ikx. k = u x,t. u( x,t)e ikx dx L. x L /2. Recall that the proof of (1) and (2) involves use of the orthogonality condition. ! Revised April 21, 2010 1:27 P! 1 Fourier Series David Radall Assume ha u( x,) is real ad iegrable If he domai is periodic, wih period L, we ca express u( x,) exacly by a Fourier series expasio: ( ) =

More information

TAKA KUSANO. laculty of Science Hrosh tlnlersty 1982) (n-l) + + Pn(t)x 0, (n-l) + + Pn(t)Y f(t,y), XR R are continuous functions.

TAKA KUSANO. laculty of Science Hrosh tlnlersty 1982) (n-l) + + Pn(t)x 0, (n-l) + + Pn(t)Y f(t,y), XR R are continuous functions. Iera. J. Mah. & Mah. Si. Vol. 6 No. 3 (1983) 559-566 559 ASYMPTOTIC RELATIOHIPS BETWEEN TWO HIGHER ORDER ORDINARY DIFFERENTIAL EQUATIONS TAKA KUSANO laculy of Sciece Hrosh llersy 1982) ABSTRACT. Some asympoic

More information

Dynamic h-index: the Hirsch index in function of time

Dynamic h-index: the Hirsch index in function of time Dyamic h-idex: he Hirsch idex i fucio of ime by L. Egghe Uiversiei Hassel (UHassel), Campus Diepebeek, Agoralaa, B-3590 Diepebeek, Belgium ad Uiversiei Awerpe (UA), Campus Drie Eike, Uiversieisplei, B-260

More information

Inference of the Second Order Autoregressive. Model with Unit Roots

Inference of the Second Order Autoregressive. Model with Unit Roots Ieraioal Mahemaical Forum Vol. 6 0 o. 5 595-604 Iferece of he Secod Order Auoregressive Model wih Ui Roos Ahmed H. Youssef Professor of Applied Saisics ad Ecoomerics Isiue of Saisical Sudies ad Research

More information

AdaBoost. AdaBoost: Introduction

AdaBoost. AdaBoost: Introduction Slides modified from: MLSS 03: Guar Räsch, Iroducio o Boosig hp://www.boosig.org : Iroducio 2 Classifiers Supervised Classifiers Liear Classifiers Percepro, Leas Squares Mehods Liear SVM Noliear Classifiers

More information

LINEAR APPROXIMATION OF THE BASELINE RBC MODEL JANUARY 29, 2013

LINEAR APPROXIMATION OF THE BASELINE RBC MODEL JANUARY 29, 2013 LINEAR APPROXIMATION OF THE BASELINE RBC MODEL JANUARY 29, 203 Iroducio LINEARIZATION OF THE RBC MODEL For f( x, y, z ) = 0, mulivariable Taylor liear expasio aroud f( x, y, z) f( x, y, z) + f ( x, y,

More information

Research Article A Generalized Nonlinear Sum-Difference Inequality of Product Form

Research Article A Generalized Nonlinear Sum-Difference Inequality of Product Form Joural of Applied Mahemaics Volume 03, Aricle ID 47585, 7 pages hp://dx.doi.org/0.55/03/47585 Research Aricle A Geeralized Noliear Sum-Differece Iequaliy of Produc Form YogZhou Qi ad Wu-Sheg Wag School

More information

2 f(x) dx = 1, 0. 2f(x 1) dx d) 1 4t t6 t. t 2 dt i)

2 f(x) dx = 1, 0. 2f(x 1) dx d) 1 4t t6 t. t 2 dt i) Mah PracTes Be sure o review Lab (ad all labs) There are los of good quesios o i a) Sae he Mea Value Theorem ad draw a graph ha illusraes b) Name a impora heorem where he Mea Value Theorem was used i he

More information

LINEAR APPROXIMATION OF THE BASELINE RBC MODEL SEPTEMBER 17, 2013

LINEAR APPROXIMATION OF THE BASELINE RBC MODEL SEPTEMBER 17, 2013 LINEAR APPROXIMATION OF THE BASELINE RBC MODEL SEPTEMBER 7, 203 Iroducio LINEARIZATION OF THE RBC MODEL For f( xyz,, ) = 0, mulivariable Taylor liear expasio aroud f( xyz,, ) f( xyz,, ) + f( xyz,, )( x

More information

BE.430 Tutorial: Linear Operator Theory and Eigenfunction Expansion

BE.430 Tutorial: Linear Operator Theory and Eigenfunction Expansion BE.43 Tuorial: Liear Operaor Theory ad Eigefucio Expasio (adaped fro Douglas Lauffeburger) 9//4 Moivaig proble I class, we ecouered parial differeial equaios describig rasie syses wih cheical diffusio.

More information

Four equations describe the dynamic solution to RBC model. Consumption-leisure efficiency condition. Consumption-investment efficiency condition

Four equations describe the dynamic solution to RBC model. Consumption-leisure efficiency condition. Consumption-investment efficiency condition LINEARIZING AND APPROXIMATING THE RBC MODEL SEPTEMBER 7, 200 For f( x, y, z ), mulivariable Taylor liear expasio aroud ( x, yz, ) f ( x, y, z) f( x, y, z) + f ( x, y, z)( x x) + f ( x, y, z)( y y) + f

More information

An interesting result about subset sums. Nitu Kitchloo. Lior Pachter. November 27, Abstract

An interesting result about subset sums. Nitu Kitchloo. Lior Pachter. November 27, Abstract A ieresig resul abou subse sums Niu Kichloo Lior Pacher November 27, 1993 Absrac We cosider he problem of deermiig he umber of subses B f1; 2; : : :; g such ha P b2b b k mod, where k is a residue class

More information

12 Getting Started With Fourier Analysis

12 Getting Started With Fourier Analysis Commuicaios Egieerig MSc - Prelimiary Readig Geig Sared Wih Fourier Aalysis Fourier aalysis is cocered wih he represeaio of sigals i erms of he sums of sie, cosie or complex oscillaio waveforms. We ll

More information

O & M Cost O & M Cost

O & M Cost O & M Cost 5/5/008 Turbie Reliabiliy, Maieace ad Faul Deecio Zhe Sog, Adrew Kusiak 39 Seamas Ceer Iowa Ciy, Iowa 54-57 adrew-kusiak@uiowa.edu Tel: 39-335-5934 Fax: 39-335-5669 hp://www.icae.uiowa.edu/~akusiak Oulie

More information

CSE 241 Algorithms and Data Structures 10/14/2015. Skip Lists

CSE 241 Algorithms and Data Structures 10/14/2015. Skip Lists CSE 41 Algorihms ad Daa Srucures 10/14/015 Skip Liss This hadou gives he skip lis mehods ha we discussed i class. A skip lis is a ordered, doublyliked lis wih some exra poiers ha allow us o jump over muliple

More information

Moment Generating Function

Moment Generating Function 1 Mome Geeraig Fucio m h mome m m m E[ ] x f ( x) dx m h ceral mome m m m E[( ) ] ( ) ( x ) f ( x) dx Mome Geeraig Fucio For a real, M () E[ e ] e k x k e p ( x ) discree x k e f ( x) dx coiuous Example

More information

F.Y. Diploma : Sem. II [AE/CH/FG/ME/PT/PG] Applied Mathematics

F.Y. Diploma : Sem. II [AE/CH/FG/ME/PT/PG] Applied Mathematics F.Y. Diploma : Sem. II [AE/CH/FG/ME/PT/PG] Applied Mahemaics Prelim Quesio Paper Soluio Q. Aemp ay FIVE of he followig : [0] Q.(a) Defie Eve ad odd fucios. [] As.: A fucio f() is said o be eve fucio if

More information

MATH 507a ASSIGNMENT 4 SOLUTIONS FALL 2018 Prof. Alexander. g (x) dx = g(b) g(0) = g(b),

MATH 507a ASSIGNMENT 4 SOLUTIONS FALL 2018 Prof. Alexander. g (x) dx = g(b) g(0) = g(b), MATH 57a ASSIGNMENT 4 SOLUTIONS FALL 28 Prof. Alexader (2.3.8)(a) Le g(x) = x/( + x) for x. The g (x) = /( + x) 2 is decreasig, so for a, b, g(a + b) g(a) = a+b a g (x) dx b so g(a + b) g(a) + g(b). Sice

More information

The Moment Approximation of the First Passage Time for the Birth Death Diffusion Process with Immigraton to a Moving Linear Barrier

The Moment Approximation of the First Passage Time for the Birth Death Diffusion Process with Immigraton to a Moving Linear Barrier America Joural of Applied Mahemaics ad Saisics, 015, Vol. 3, No. 5, 184-189 Available olie a hp://pubs.sciepub.com/ajams/3/5/ Sciece ad Educaio Publishig DOI:10.1691/ajams-3-5- The Mome Approximaio of

More information

Electrical Engineering Department Network Lab.

Electrical Engineering Department Network Lab. Par:- Elecrical Egieerig Deparme Nework Lab. Deermiaio of differe parameers of -por eworks ad verificaio of heir ierrelaio ships. Objecive: - To deermie Y, ad ABD parameers of sigle ad cascaded wo Por

More information

Solutions to Problems 3, Level 4

Solutions to Problems 3, Level 4 Soluios o Problems 3, Level 4 23 Improve he resul of Quesio 3 whe l. i Use log log o prove ha for real >, log ( {}log + 2 d log+ P ( + P ( d 2. Here P ( is defied i Quesio, ad parial iegraio has bee used.

More information

B. Maddah INDE 504 Simulation 09/02/17

B. Maddah INDE 504 Simulation 09/02/17 B. Maddah INDE 54 Simulaio 9/2/7 Queueig Primer Wha is a queueig sysem? A queueig sysem cosiss of servers (resources) ha provide service o cusomers (eiies). A Cusomer requesig service will sar service

More information

A Note on Random k-sat for Moderately Growing k

A Note on Random k-sat for Moderately Growing k A Noe o Radom k-sat for Moderaely Growig k Ju Liu LMIB ad School of Mahemaics ad Sysems Sciece, Beihag Uiversiy, Beijig, 100191, P.R. Chia juliu@smss.buaa.edu.c Zogsheg Gao LMIB ad School of Mahemaics

More information

Solutions to selected problems from the midterm exam Math 222 Winter 2015

Solutions to selected problems from the midterm exam Math 222 Winter 2015 Soluios o seleced problems from he miderm eam Mah Wier 5. Derive he Maclauri series for he followig fucios. (cf. Pracice Problem 4 log( + (a L( d. Soluio: We have he Maclauri series log( + + 3 3 4 4 +...,

More information

C(p, ) 13 N. Nuclear reactions generate energy create new isotopes and elements. Notation for stellar rates: p 12

C(p, ) 13 N. Nuclear reactions generate energy create new isotopes and elements. Notation for stellar rates: p 12 Iroducio o sellar reacio raes Nuclear reacios geerae eergy creae ew isoopes ad elemes Noaio for sellar raes: p C 3 N C(p,) 3 N The heavier arge ucleus (Lab: arge) he ligher icomig projecile (Lab: beam)

More information

VIM for Determining Unknown Source Parameter in Parabolic Equations

VIM for Determining Unknown Source Parameter in Parabolic Equations ISSN 1746-7659, Eglad, UK Joural of Iformaio ad Compuig Sciece Vol. 11, No., 16, pp. 93-1 VIM for Deermiig Uko Source Parameer i Parabolic Equaios V. Eskadari *ad M. Hedavad Educaio ad Traiig, Dourod,

More information

Page 1. Before-After Control-Impact (BACI) Power Analysis For Several Related Populations. Richard A. Hinrichsen. March 3, 2010

Page 1. Before-After Control-Impact (BACI) Power Analysis For Several Related Populations. Richard A. Hinrichsen. March 3, 2010 Page Before-Afer Corol-Impac BACI Power Aalysis For Several Relaed Populaios Richard A. Hirichse March 3, Cavea: This eperimeal desig ool is for a idealized power aalysis buil upo several simplifyig assumpios

More information

A Note on Prediction with Misspecified Models

A Note on Prediction with Misspecified Models ITB J. Sci., Vol. 44 A, No. 3,, 7-9 7 A Noe o Predicio wih Misspecified Models Khresha Syuhada Saisics Research Divisio, Faculy of Mahemaics ad Naural Scieces, Isiu Tekologi Badug, Jala Gaesa Badug, Jawa

More information

CLOSED FORM EVALUATION OF RESTRICTED SUMS CONTAINING SQUARES OF FIBONOMIAL COEFFICIENTS

CLOSED FORM EVALUATION OF RESTRICTED SUMS CONTAINING SQUARES OF FIBONOMIAL COEFFICIENTS PB Sci Bull, Series A, Vol 78, Iss 4, 2016 ISSN 1223-7027 CLOSED FORM EVALATION OF RESTRICTED SMS CONTAINING SQARES OF FIBONOMIAL COEFFICIENTS Emrah Kılıc 1, Helmu Prodiger 2 We give a sysemaic approach

More information

Fresnel Dragging Explained

Fresnel Dragging Explained Fresel Draggig Explaied 07/05/008 Decla Traill Decla@espace.e.au The Fresel Draggig Coefficie required o explai he resul of he Fizeau experime ca be easily explaied by usig he priciples of Eergy Field

More information

A Novel Approach for Solving Burger s Equation

A Novel Approach for Solving Burger s Equation Available a hp://pvamu.edu/aam Appl. Appl. Mah. ISSN: 93-9466 Vol. 9, Issue (December 4), pp. 54-55 Applicaios ad Applied Mahemaics: A Ieraioal Joural (AAM) A Novel Approach for Solvig Burger s Equaio

More information

Lecture 15: Three-tank Mixing and Lead Poisoning

Lecture 15: Three-tank Mixing and Lead Poisoning Lecure 15: Three-ak Miig ad Lead Poisoig Eigevalues ad eigevecors will be used o fid he soluio of a sysem for ukow fucios ha saisfy differeial equaios The ukow fucios will be wrie as a 1 colum vecor [

More information

Using Linnik's Identity to Approximate the Prime Counting Function with the Logarithmic Integral

Using Linnik's Identity to Approximate the Prime Counting Function with the Logarithmic Integral Usig Lii's Ideiy o Approimae he Prime Couig Fucio wih he Logarihmic Iegral Naha McKezie /26/2 aha@icecreambreafas.com Summary:This paper will show ha summig Lii's ideiy from 2 o ad arragig erms i a cerai

More information

Problems and Solutions for Section 3.2 (3.15 through 3.25)

Problems and Solutions for Section 3.2 (3.15 through 3.25) 3-7 Problems ad Soluios for Secio 3 35 hrough 35 35 Calculae he respose of a overdamped sigle-degree-of-freedom sysem o a arbirary o-periodic exciaio Soluio: From Equaio 3: x = # F! h "! d! For a overdamped

More information

K3 p K2 p Kp 0 p 2 p 3 p

K3 p K2 p Kp 0 p 2 p 3 p Mah 80-00 Mo Ar 0 Chaer 9 Fourier Series ad alicaios o differeial equaios (ad arial differeial equaios) 9.-9. Fourier series defiiio ad covergece. The idea of Fourier series is relaed o he liear algebra

More information

Basic Results in Functional Analysis

Basic Results in Functional Analysis Preared by: F.. ewis Udaed: Suday, Augus 7, 4 Basic Resuls i Fucioal Aalysis f ( ): X Y is coiuous o X if X, (, ) z f( z) f( ) f ( ): X Y is uiformly coiuous o X if i is coiuous ad ( ) does o deed o. f

More information

Clock Skew and Signal Representation

Clock Skew and Signal Representation Clock Skew ad Sigal Represeaio Ch. 7 IBM Power 4 Chip 0/7/004 08 frequecy domai Program Iroducio ad moivaio Sequeial circuis, clock imig, Basic ools for frequecy domai aalysis Fourier series sigal represeaio

More information

Approximately Quasi Inner Generalized Dynamics on Modules. { } t t R

Approximately Quasi Inner Generalized Dynamics on Modules. { } t t R Joural of Scieces, Islamic epublic of Ira 23(3): 245-25 (22) Uiversiy of Tehra, ISSN 6-4 hp://jscieces.u.ac.ir Approximaely Quasi Ier Geeralized Dyamics o Modules M. Mosadeq, M. Hassai, ad A. Nikam Deparme

More information

L-functions and Class Numbers

L-functions and Class Numbers L-fucios ad Class Numbers Sude Number Theory Semiar S. M.-C. 4 Sepember 05 We follow Romyar Sharifi s Noes o Iwasawa Theory, wih some help from Neukirch s Algebraic Number Theory. L-fucios of Dirichle

More information

Stochastic Processes Adopted From p Chapter 9 Probability, Random Variables and Stochastic Processes, 4th Edition A. Papoulis and S.

Stochastic Processes Adopted From p Chapter 9 Probability, Random Variables and Stochastic Processes, 4th Edition A. Papoulis and S. Sochasic Processes Adoped From p Chaper 9 Probabiliy, adom Variables ad Sochasic Processes, 4h Ediio A. Papoulis ad S. Pillai 9. Sochasic Processes Iroducio Le deoe he radom oucome of a experime. To every

More information

Principles of Communications Lecture 1: Signals and Systems. Chih-Wei Liu 劉志尉 National Chiao Tung University

Principles of Communications Lecture 1: Signals and Systems. Chih-Wei Liu 劉志尉 National Chiao Tung University Priciples of Commuicaios Lecure : Sigals ad Sysems Chih-Wei Liu 劉志尉 Naioal Chiao ug Uiversiy cwliu@wis.ee.cu.edu.w Oulies Sigal Models & Classificaios Sigal Space & Orhogoal Basis Fourier Series &rasform

More information

Chapter 2: Time-Domain Representations of Linear Time-Invariant Systems. Chih-Wei Liu

Chapter 2: Time-Domain Representations of Linear Time-Invariant Systems. Chih-Wei Liu Caper : Time-Domai Represeaios of Liear Time-Ivaria Sysems Ci-Wei Liu Oulie Iroucio Te Covoluio Sum Covoluio Sum Evaluaio Proceure Te Covoluio Iegral Covoluio Iegral Evaluaio Proceure Iercoecios of LTI

More information