Decentralized Double Stochastic Averaging Gradient
|
|
- Christian Dawson
- 5 years ago
- Views:
Transcription
1 Deceralized Double Sochasic Averagig Gradie Arya Mokhari ad Alejadro Ribeiro Deparme of Elecrical ad Sysems Egieerig, Uiversiy of Pesylvaia Absrac This paper cosiders cove opimizaio problems where odes of a ework have access o summads of a global objecive fucio. Each of hese local objecives is furher assumed o be a average of a fiie se of fucios. The moivaio for his seup is o solve large scale machie learig problems where elemes of he raiig se are disribued o muliple compuaioal elemes. The deceralized double sochasic averagig gradie (DSA) algorihm is proposed as a soluio aleraive ha relies o: (i) The use of local sochasic averagig gradies isead of local full gradies. (ii) Deermiaio of desce seps as differeces of cosecuive sochasic averagig gradies. The algorihm is show o approach he opimal argume a a liear rae. This is i coras o all oher available mehods for disribued sochasic opimizaio ha coverge a subliear raes. Numerical eperimes verify liear covergece of DSA ad illusrae is advaages relaive o hese oher aleraives. I. INTRODUCTION Cosider a variable R p ad a coeced ework of size N where each ode has access o a local objecive fucio f : R p R. The local objecive fucio f () is defied as he average of fucios f,i () ha ca be idividually evaluaed a ode. Ages cooperae o solve he global opimizaio := argmi f () = argmi q i= f,i (). () The formulaio i () models large scale machie learig problems where elemes of he raiig se are disribued o muliple compuaioal elemes ad represes a opimal classifier, 2. Aalogous formulaios are also of ieres i deceralized corol 3, 4 ad sesor eworks 5 7. Our ieres here is i solvig () wih a mehod ha is disribued odes operae o heir local fucios ad commuicae wih eighbors ad sochasic odes uilize oly oe ou of he fucios f,i o deermie a desce direcio. Disribued mehods germae o his paper are deceralized gradie desce (DGD) ad is varias 8, as well as he eac firs order algorihm (EXTRA) 2. A issue wih he former ha is solved by he laer is he lack of liear covergece rae guaraees ha EXTRA achieves by usig ieraios ha rely o iformaio of wo cosecuive seps. Sochasic opimizaio mehods relaed o he proposal i his paper are sochasic gradie desce 3 6 ad sochasic averagig gradie mehods 7, 8. As i he case of disribued opimizaio, he former have subliear covergece raes bu he laer have liear covergece raes. They achieve hese liear raes by usig icremeal gradies o reduce he sochasic gradie oise. This reducio follows from a memory rade ha permis maiaiig a average of pas gradies i which oly oe erm is updaed per ieraio. The coribuio of his paper is o develop he deceralized double sochasic averagig gradie (DSA) mehod, a ovel deceralized sochasic algorihm for solvig (). The mehod eplois a ew ierpreaio of EXTRA as a saddle poi mehod (Secio II) ad uses sochasic averagig gradies i lieu of gradies (Secio III). The proposed mehod coverges liearly o he opimal argume i epecaio (Secio IV). This is i coras o all oher disribued sochasic mehods o solve () ha coverge a subliear raes. Numerical resuls verify ha DSA is he oly sochasic deceralized algorihm wih liear covergece rae (Secio V). Proofs of resuls i his paper are available i 9. II. DECENTRALIZED DOUBLE GRADIENT DESCENT Cosider a coeced ework ha coais N odes such ha each ode ca oly commuicae wih odes i is eighborhood N. Defie R p as a local copy of he variable ha is kep a ode. I deceralized opimizaio, odes ry o miimize heir local fucios f ( ) while esurig ha heir local variables coicide wih he variables m of all eighbors m N which, give ha he ework is coeced, esures ha he variables of all odes are he same ad reders he problem equivale o (). DGD is a well kow mehod for deceralized opimizaio ha relies o he iroducio of oegaive weighs w ij 0 ha are o ull if ad oly if m = or if m N. Leig N be a discree ime ide ad α a give sepsize, DGD is defied by he recursio + = w m m α f ( ), =,..., N. (2) m= Sice w m = 0 whe m ad m / N, i follows from (2) ha ode updaes by performig a average over he variables m of is eighbors m N ad is ow, followed by desce hrough he egaive local gradie f ( ). If a cosa sepsize is used, DGD ieraes approach a eighborhood of he opimal argume of () bu do coverge eacly. To achieve eac covergece dimiishig sepsizes are used bu he resulig covergece rae is subliear 8. EXTRA resolves eiher of hese issues by miig wo cosecuive DGD ieraios wih differe weigh marices ad opposie sigs. To be precise, iroduce a secod se of weighs w m wih he same properies as he weighs w m ad defie EXTRA hrough he recursio + = + w m m m= m= w m m (3) α f ( ) f ( ), =,..., N. Observe ha (3) is well defied for > 0. For = 0 we uilize he regular DGD ieraio i (2). I he omeclaure of his paper we say ha EXTRA performs a deceralized double gradie desce sep because i operaes i deceralized maer while uilizig a differece of wo gradies as desce direcio. Mior modificaio as i is, he use of his gradie differece i lieu of
2 simple gradies edows era wih eac liear covergece o he opimal argume 2. To udersad he raioaliy behid he EXTRA updae, we defie marices ad vecors o rewrie updaes i (3) for differe odes as a sigle equaio. To do so, defie he vecor := ;... ; N R Np which cocaeaes he local ieraes, ad he aggregae fucio f : R Np R as f () = f(,..., N ) := f ( ). (4) Moreover, Cosider he marices W R N N ad W R N N formed by compoes w m ad w m, respecively. Defie he marices Z := W I R Np Np ad Z := W I R Np Np as he Kroecker producs of he weigh marices W R N N ad W R N N by he ideiy mari I R p p, respecively. Cosiderig hese defiiios, we ca rewrie he EXTRA s updae for > 0 i (3) as + = (I + Z) Z α f( ) f( ), (5) ad he iiial sep as = Z 0 α f( 0 ). (6) By summig up he updaes i (5) ad (6) from sep 0 o ad usig he elescopic cacellaio we obai ha + = Z α f( ) ( Z Z) s. (7) s=0 We iroduce a primal-dual ierpreaio of he updae i (7) by defiig he sequece of vecors v = s=0 ( Z Z) /2 s as he accumulaio of variables dissimilariies i differe odes over ime. Noe ha if compoes of he vecor s are equal o each oher, i.e., s = = s N, he correspodig erm of he sum i he defiiio of vecor v is ull, i.e. ( Z Z) /2 s = 0. Cosiderig he defiiio of v we ca rewrie (7) as + = α f( ) + α (I Z) + α ( Z Z) /2 v. (8) Furher, based o he defiiio of sequece v = s=0 ( Z Z) /2 s we ca wrie v + as v + = v + α α ( Z Z) /2 +. (9) Cosider as he primal variable ad v as he dual variable. The, he EXTRA updae is equivale o a saddle poi mehod wih sepsize α for solvig he Lagragia L(, v) = f() + α vt ( Z Z) /2 + 2α T (I Z), (0) where he he acual Lagragia is augmeed by he quadraic erm (/2α) T (I Z). Observe ha he opimizaio problem wih he augmeed Lagragia i (0) is mi f() s.. α ( Z Z) /2 = 0. () Observig ha ull(( Z Z) /2 ) = ull( Z Z) = spa{ N I p }, he cosrai i () is equivale o = = N. Moreover, he defiiio of fucio f() i (4) shows ha he objecives of problems () ad () are also ideical. Hece, EXTRA is a saddle poi mehod ha solves () which is equivale o (). Cosiderig he eac ad liear covergece of saddle poi mehods, he covergece properies of EXTRA are jusified. III. DECENTRALIZED DOUBLE STOCHASTIC AVERAGING GRADIENT Recall he defiiios of he local fucios f ( ) ad he isaaeous local fucios f,i ( ) available a ode. To impleme EXTRA as i (3) each ode compues he full gradie of is local objecive fucio f ( ) = (/ ) i= f,i( ) which is compuaioally epesive whe he umber of isaaeous fucios is eremely large. To resolve his issue he local objecive gradies ca be subsiued by heir sochasic approimaios. The simples approach for approimaig he local objecive fucios gradie f ( ) = (/ ) i= f,i( ) is choosig a isaaeous fucio f,i ( ) radomly ad usig is gradie f,i ( ) as a ubiased esimae of he gradie f ( ) = (/ ) i= f,i( ). To be more precise, defie vecor θ = θ :... ; θ N {,..., q } {,..., q N } as a radom vecor where each compoe θ {,..., } deermies he associaed isaaeous fucio f,θ ( ) ha ode uses for gradie approimaio. To be more precise, ode i lieu of compuig he local fucio gradie f () for updaig he variable, approimaes i by f,θ ( ). However, sochasic gradies lead o a algorihm wih lower compuaio compleiy, he oise of gradie approimaio avoids eac covergece wih cosa sepsize as show for sochasic gradie desce i ceralized opimizaio. We sudy his observaio i Secio V. To overcome he oise of gradie approimaio we use he idea of ubiased sochasic averagig gradie as iroduced i 8. We iroduce he auiliary vecors φ,i R p correspodig o i-h isaaeous fucio of ode which keeps rack of he ierae for he las sep ha i-h isaaeous fucio f,i is chose a ode. To be more precise, if he ide ideifier a ime for ode is θ = i he he correspodig auiliary vecor = ad is correspodig isaaeous fucio gradie f,i (φ,i) which is sored i a memory is replaced by f,i ( ). All he oher auiliary vecors φ,j for j i ad heir correspodig isaaeous gradies remai φ,i is updaed as φ +,i uchaged, i.e. φ +,j = φ,j ad f,j (φ +,j ) = f,j(φ,j). By sorig he auiliary variables gradies f,i (φ,i), we ca defie a ubiased esimae of he local gradie f ( ) as ĝ := f,θ ( ) f,θ (φ,θ )+ i= f,i (φ,i). (2) Noice ha he sochasic approimaio ĝ is a ubiased esimae of he local gradie f ( ), i.e., E ĝ F = f ( ). The proposed sochasic averagig gradie i (2) vaishes he oise of gradie approimaio. To be more precise, as ime progresses he auiliary variables φ,i approach o a eighborhood of he opimal variable, sice hey all ge updaed over ime wih a high probabiliy. Therefore, roughly speakig we ca wrie φ,i. This propery implies ha he sochasic gradie i (2) ca be approimaed by ĝ (/ ) i= f,i(φ,i) f ( ). Therefore, he advaage of usig sochasic approimaio i (2) is he fac ha he oise of sochasic gradies is dimiishig whe he sequece is close o covergece, while for he aive approimaio f,θ ( ) he oise of sochasic approimaio ever vaishes. We iroduce Deceralized Double sochasic averagig gradie (DSA) as a sochasic versio of EXTRA ha approimaes he local gradies by heir sochasic averagig approimaios
3 Algorihm DSA algorihm a ode Require: Vecor 0 ad sored gradies f,i(φ 0,i ) wih φ0,i = 0. : for = 0,, 2,... do 2: Echage variable wih eighborig odes m N. 3: Choose θ uiformly radom from he se {,..., }. 4: Compue ad sore sochasic averagig gradie ĝ = f,θ ( ) f,θ (φ,θ ) + q f,i(φ,i q ) i= 5: Se φ + =,θ ad sore gradie f,θ (φ + i he,θ) able replacig f,θ (φ,θ ). Oher vecors of he able remai uchaged, i.e. f,j(φ +,j ) = f,j(φ,j) for j θ. 6: Updae primal variable as 7: if = 0 he 8: + = 9: else 0: + = + : ed if 2: ed for w m + αĝ. w m ŵ m α ĝ ĝ. as iroduced i (2). The DSA updae for > 0 is give by + = + w m m m= ad he iiial sep is defied as = m= w m m α ĝ ĝ, (3) w m 0 m α ĝ. 0 (4) m= To wrie he DSA updae for all he odes i oe equaio, defie he vecor ĝ := ĝ ;... ; ĝ N RNp which coais all he local sochasic averagig gradies a sep. Cosiderig his defiiio he updaes for seps > 0 i (3) ca be simplified as + = (I + Z) Z α ĝ ĝ, (5) ad he iiial updaes i (4) are equivale o = Z 0 α ĝ 0. (6) Comparig he DSA updaes i (5) ad (6) wih EXTRA seps i (5) ad (6) shows ha DSA is differe from EXTRA i usig sochasic averagig gradies ĝ i lieu of full gradies f( ). Recall ha EXTRA is a saddle poi mehod for solvig (). Therefore, DSA is a sochasic saddle poi mehod ha solves problem () where he primal variable is updaed as + = αĝ (I Z) ( Z Z) /2 v, (7) ad he dual variable v is updaed as v + = v + ( Z Z) /2 +. (8) Noice ha he iiial primal variable 0 R Np is a arbirary vecor, while accordig o he defiiio v = s=0 ( Z Z) /2 s he iiial dual vecor is se as v 0 = ( Z Z) /2 0. To impleme DSA we use he updae i (5) isead of usig he primal-dual updaes i (7) ad (8). The laer requires echage of he boh primal ad dual v variables, while for he former oly echage of he primal variables is required. The DSA algorihm is summarized i Algorihm. The updae of DSA for = 0 ad > 0 are implemeed i Seps 8 ad 0, respecively. Seps 8 ad 0 require access o he local ieraes m of he eighborig odes m N which are colleced i Sep 2. Furher, implemeaio of he DSA updae requires sochasic gradies ĝ ad ĝ which are compued i Sep 5 of ieraios ad, respecively. I Sep 3 he ide θ is chose radomly o disiguish he isaaeous fucio f,θ ha we use is gradies a pois ad φ,θ for compuig he sochasic averagig gradie i Sep 4. The able of auiliary variables gradies is updaed i Sep 5 by replacig f,θ (φ,θ ) by f,θ ( ), while he oher vecors remai uchaged. IV. CONVERGENCE ANALYSIS Our goal here is o show ha as ime progresses he sequece of ieraes approaches he opimal argume. I provig his resul for he DSA algorihm we make he followig assumpios. Assumpio. The wigh marices W ad W saisfy (a) If m ad m / N, he w m = w m = 0. (b) W = W T ad W = W T. (c) ull{ W W} = spa{} ad ull{i W} spa{}. (d) 0 W ad W W (I + W)/2. Assumpio 2. The isaaeous local fucios f,i ( ) are differeiable ad srogly cove wih parameer µ. Assumpio 3. The isaaeous local fucios gradies f,i are Lipschiz coiuous wih parameer L, f,i (a) f,i (b) L a b a, b R p. (9) The codiios imposed by Assumpio (a) o he eries of he weigh marices W ad W imply ha odes oly have access o he local ad eighborig iformaio. Furher, we assume he assiged weighs are symmeric for boh weigh marices W ad W as meioed i Assumpio (b). Codiios o he specral properies of marices W ad W i Assumpios (c) ad (d) imply ha ull{i W} = spa{} see Proposiio Assumpio 2 implies ha he local fucios f ( ) ad he global cos fucio f() = N f ( ) are srogly cove wih parameer µ. Likewise, he Lipschiz coiuiy of he local isaaeous gradies f,i ( ) eforces Lipschiz coiuiy of he local fucios gradies f ( ) ad he aggregae fucio gradies f(). Defie 0 < γ ad Γ < as he smalles ad larges eigevalues of Z, respecively. Likewise, defie γ as he smalles o-zero eigevalue of he mari Z Z ad Γ as he larges eigevalue of Z Z. Furher, defie vecors u, u R 2Np ad mari G R 2Np 2Np as u :=, u :=, G = v v Z 0. (20) 0 I The vecor u R 2Np cocaeaes he opimal primal ad dual variables ad he vecor u R 2Np coais primal ad dual ieraes a sep. Furher, G R 2Np 2Np is a block diagoal posiive defiie mari. We sudy he covergece properies of he weighed orm u u 2 G which is equivale o (u u ) T G(u u ). Our goal is o show ha he sequece u u 2 G coverges liearly o ull. To do his we show liear covergece of a Lyapuov fucio of he sequece u u 2 G.
4 The Lyapuov fucio is defied as u u 2 G + cp where p := i= f,i (φ,i) f ( ) i= f,i ( ) T (φ,i ), (2) ad c > 0 is a posiive cosa. Noice ha based o he srog coveiy of he local isaaeous fucios f,i, each erm f,i (φ,i) f,i ( ) f,i ( ) T (φ,i ) is posiive ad as a resul he sequece p defied i (2) is always posiive. To prove liear covergece of he sequece u u 2 G + cp we firs show a upper boud for he epeced error E u + u 2 G F i erms of u u 2 G ad some parameers ha capure he opimaliy gap. Lemma. Cosider he DSA algorihm as defied i (2)-(8). Furher, recall he defiiios of p i (2) ad u, u, ad G i (20). If Assumpios -3 hold rue, he for ay posiive cosas η, ρ > 0 we ca wrie E u + u 2 G F u u 2 G (22) E + 2 Z α(η+ρ)i F 2E + F 2I+Z 2 Z E v + v 2 F + 4αL ρ p αc 0 2, where C 0 = (2µ 2 /L) (L 2 /η) (2(L 2 µ 2 ))/ρ. Likewise, we provide a upper boud for he oher par of he Lyapuov fucio a ime + which is p + i erms of p ad some parameers ha capure opimaliy gap. This boud is sudied i he followig lemma. Lemma 2. Cosider he DSA algorihm as defied i (2)-(8) ad he defiiio of p i (2). Furher, defie q mi ad q ma as he smalles ad larges values for he size of isaaeous fucios a a ode, respecively. If Assumpios -3 hold rue, he for all > 0 E p + F ( ) p + q ma L 2q mi 2. (23) Combiig he resuls i Lemmaa ad 2 we ca show ha he epeced Lyapuov fucio E u + u 2 G + c p+ F is sricly smaller ha is previous value u u 2 G + c p. Theorem. Cosider he DSA algorihm as defied i (2)- (8). Furher, recall he defiiios of p i (2) ad u, u, ad G i (20). Moreover, cosider he resuls i (22) ad (23). If Assumpios -3 hold rue ad he sepsize α ad he parameer c are chose properly he here eis 0 < δ < such ha E u + u 2 G + c p + F ( δ) ( u u 2 G + c p ). (24) The codiios o α ad c, ad he eplici epressio of δ are provided i 9. The iequaliy i (24) shows ha he epeced value of he sequece u + u 2 G + c p+ give he observaios uil sep is sricly smaller ha he previous ierae u u 2 G +cp. By akig he epeced value wih respec o he iiial field E. F 0 = E. ad applyig he implied iequaliy recursively we obai ha E u u 2 G + c p ( δ) ( u 0 u 2 G + c p 0). (25) Accordig o (25), he sequece u u 2 G + c p coverges liearly o ull i epecaio. Noice ha he orm u u 2 G is equal o 2 Z + v v 2. Hece, he iequaliy 2 Z u u 2 G holds. Moreover, he sequece p is always o-egaive. Cosiderig hese wo observaios he iequaliy 2 Z u u 2 G + c p holds rue. Cosiderig his iequaliy ad he epressio i (25), ad observig ha he erm 2 Z is lower bouded by γ 2, we ca wrie he followig corollary. Corollary. Cosider he DSA algorihm as defied i (2)-(8). Recall he defiiios of p i (2) ad u, u, ad G i (20). Furher, recall γ as he smalles eigevalue of Z. If Assumpios -3 hold rue, he here eis a cosa 0 < δ < such ha E 2 ( δ) ( u 0 u 2 G + c p0). γ (26) Corollary saes ha E 2 liearly coverges o ull. Noe ha he sequece E 2 is o ecessarily decreasig as he sequece E u u 2 G + c p is. V. NUMERICAL ANALYSIS We umerically sudy he performace of he DSA algorihm for solvig a logisic regressio problem. Cosider q = N raiig pois where each ode has access o of hem. The raiig pois a ode are deoed by s i R p for i =,..., wih he associaed labels l i {, }. The goal is o solve he logisic regressio problem := argmi R p i= log + ep( l i s T i), (27) where he regularizaio erm (/2) 2 is added o avoid overfiig he raiig model. The problem i (27) ca be wrie i he form of () by defiig he local objecive fucios f as f () = 2N 2 + i= ad he isaaeous local fucios f,i as log + ep( l i s T i), (28) f,i () = 2N 2 + log ( + ep ( l i s T i )), (29) for all i =,...,. I our eperimes we use a syheic daase where compoes of he feaure vecors s i wih label l i = are geeraed from a ormal disribuio wih mea µ ad sadard deviaio σ +, while he disribuio of he sample pois wih label l i = is ormal wih mea µ ad sadard deviaio σ. The edges bewee odes are geeraed radomly wih probabiliy p c. The weigh mari W is geeraed usig he Laplacia mari L of he ework as W = I L/τ, where τ > (/2) ma (L). The covergece error is defied ad compued as e = 2. We se he oal umber of sample pois q = 500, feaure vecors dimesio p = 2, regularizaio parameer = 0 4, probabiliy of eisece of a edge p c = 0.3, ad τ = (2/3) ma (L). To make he daase o liearly separable we se he mea value as µ = 2 ad he sadard deviaios as σ + = σ = 2. We provide a compariso of DSA wih respec o DGD, EX- TRA, sochasic EXTRA, ad deceralized SAGA. The sochasic EXTRA is a sochasic versio of EXTRA ha uses aive sochasic gradies isead of gradies. DSA is differe form sochasic EXTRA sice i uses sochasic averagig gradies.
5 Radom e. pc = 0.2 Radom e. pc = 0.3 Complee e. Cycle Lie Sar error e EXTRA α = DSA α = DGD α = 0 3 D-SAG α = 0 3 DGD α = 0 2 D-SAG α = 0 2 so-extra α = 0 3 so-extra α = 0 2 error e Number of ieraios Number of ieraios Fig.. Covergece pahs of DSA, EXTRA, DGD, Sochasic EXTRA, ad Deceralized SAGA wih cosa sepsizes. Relaive disace o opimaliy e = 2 is show wih respec o he umber ieraios. DSA ad EXTRA coverge liearly o he opimum, while DGD, Sochasic EXTRA, ad Deceralized SAGA coverge o a eighborhood of he opimal soluio. Smaller choice of sepsize leads o more accurae covergece for hese algorihms. The deceralized SAGA mehod is a sochasic versio of DGD algorihm ha uses sochasic averagig gradies isead of gradies which is a aive approach for developig a deceralized versio of he SAGA algorihm. I our eperimes we use W = (I + W)/2 for EXTRA, sochasic EXTRA, ad DSA. Fig. illusraes he covergece pahs of DSA, EXTRA, DGD, Sochasic EXTRA, ad Deceralized SAGA wih cosa sep sizes for N = 20 odes. For EXTRA ad DSA differe sepsize are chose ad he bes performace for EXTRA ad DSA are achieved by α = ad α = 5 0 3, respecively. As show i Fig., DSA is he oly sochasic algorihm ha achieves liear covergece. Deceralized SAGA afer couple of ieraios achieves he performace of DGD ad hey boh ca o achieve eac covergece. By choosig smaller sepsize α = 0 3 hey reach more accurae covergece relaive o sepsize α = 0 2, however, he speed of covergece is slower for he smaller sepsize. Sochasic EXTRA also suffers from ieac covergece, bu for a differe reaso. DGD ad deceralized SAGA have ieac covergece sice hey solve a pealy versio of (), while sochasic EXTRA ca o reach he opimal soluio sice he oise of sochasic gradie is o vaishig. DSA resolves boh issues by combiig he idea of sochasic averagig gradies o corol he oise of sochasic gradies ad usig he double deceralized desce idea o solve he correc opimizaio problem. The covergece rae of EXTRA is faser ha he oe for DSA i erms of umber of ieraios, however, he compleiy of EXTRA is higher ha DSA. Hece, we also compare performaces of hese algorihms i erms of umber of processed feaure vecors. For isace, DSA requires 400 ieraios or equivalely 400 feaure vecors o achieve error e = 0 7, while o achieve he same accuracy EXTRA requires 60 ieraios which is equivale o processig = 440 feaure vecors. The differece ca be more sigifica by icreasig he umber of isaaeous fucios. We also sudy he performace of DSA i differe ework opologies. We keep he parameers i Fig. ecep we chage he size of ework o N = 00 which implies each ode has q i = 5 sample pois. The liear covergece of DSA for radom eworks, complee graph, cycle, lie ad sar are show i Fig. 2. As we epec for he opologies ha he graph is more coeced ad he diameer is smaller DSA coverges faser. Fig. 2. Covergece pahs of DSA for differe ework opologies. Relaive disace o opimaliy e = 2 is show wih respec o he umber ieraios. DSA coverges faser as he ework coeciviy icreases. REFERENCES R. Bekkerma, M. Bileko, ad J. Lagford, Scalig up machie learig: Parallel ad disribued approaches. Cambridge Uiversiy Press, K. Tsiaos, S. Lawlor, ad M. Rabba, Cosesus-based disribued opimizaio: Pracical issues ad applicaios i large-scale machie learig, I: Proceedigs of Allero Coferece o Commuicaio, Corol, ad Compuig, Y. Cao, W. Yu, W. Re, ad G. Che, A overview of rece progress i he sudy of disribued muli-age coordiaio, IEEE Trasacios o Idusrial Iformaics, vol. 9, pp , C. G. Lopes ad A. H. Sayed, Diffusio leas-mea squares over adapive eworks: Formulaio ad performace aalysis, IEEE Trasacios o Sigal Processig, vol. 56, o. 7, pp , July I. Schizas, A. Ribeiro, ad G. Giaakis, Cosesus i ad hoc wss wih oisy liks - par i: Disribued esimaio of deermiisic sigals, IEEE Trasacios o Sigal Processig, vol. 56, pp , U. A. Kha, S. Kar, ad J. M. Moura, Dilad: A algorihm for disribued sesor localizaio wih oisy disace measuremes, IEEE Trasacios o Sigal Processig, vol. 58, o. 3, pp , M. Rabba ad R. Nowak, Disribued opimizaio i sesor eworks, proceedigs of he 3rd ieraioal symposium o Iformaio processig i sesor eworks, pp , ACM, A. Nedic ad A. Ozdaglar, Disribued subgradie mehods for muliage opimizaio, IEEE Trasacios o Auomaic Corol, vol. 54, pp. 48 6, D. Jakoveic, J. Xavier, ad J. Moura, Fas disribued gradie mehods, IEEE Trasacios o Auomaic Corol, vol. 59, pp. 3 46, K. Yua, Q. Lig, ad W. Yi, O he covergece of deceralized gradie desce, arxiv prepri arxiv, , 203. A. Mokhari, Q. Lig, ad A. Ribeiro, Nework ewo, i Proc. Asilomar Cof. o Sigals Sysems Compuers, vol. (o appear). Pacific Grove CA, November , available a hp://ariv.org/pdf/ pdf. 2 W. Shi, Q. Lig, G. Wu, ad W. Yi, Era: A eac firs-order algorihm for deceralized cosesus opimizaio, arxiv prepri arxiv, J. N. Tsisiklis, D. P. Bersekas, M. Ahas e al., Disribued asychroous deermiisic ad sochasic gradie opimizaio algorihms, IEEE rasacios o auomaic corol, vol. 3, o. 9, pp , J. Duchi, A. Agarwal, ad M. Waiwrigh, Dual averagig for disribued opimizaio: Covergece aalysis ad ework scalig, IEEE Trasacios o Auomaic Corol, vol. 57, pp , S. S. Ram, A. Nedic, ad V. V. Veeravalli, Icremeal sochasic subgradie algorihms for cove opimizaio, SIAM Joural o Opimizaio, vol. 20, o. 2, pp , S. S. Ram, A. Nedić, ad V. V. Veeravalli, Disribued sochasic subgradie projecio algorihms for cove opimizaio, Joural of opimizaio heory ad applicaios, vol. 47, o. 3, pp , M. Schmid, N. L. Rou, ad F. Bach, Miimizig fiie sums wih he sochasic average gradie, arxiv prepri arxiv: , A. Defazio, F. Bach, ad S. Lacose-Julie, Saga: A fas icremeal gradie mehod wih suppor for o-srogly cove composie objecives, i Advaces i Neural Iformaio Processig Sysems, 204, pp A. Mokhari ad A. Ribeiro, Deceralized double sochasic averagig gradie, 205, Olie:hp:// aryam/wiki/dsa.pdf.
Supplement for SADAGRAD: Strongly Adaptive Stochastic Gradient Methods"
Suppleme for SADAGRAD: Srogly Adapive Sochasic Gradie Mehods" Zaiyi Che * 1 Yi Xu * Ehog Che 1 iabao Yag 1. Proof of Proposiio 1 Proposiio 1. Le ɛ > 0 be fixed, H 0 γi, γ g, EF (w 1 ) F (w ) ɛ 0 ad ieraio
More information1 Notes on Little s Law (l = λw)
Copyrigh c 26 by Karl Sigma Noes o Lile s Law (l λw) We cosider here a famous ad very useful law i queueig heory called Lile s Law, also kow as l λw, which assers ha he ime average umber of cusomers i
More informationComparison between Fourier and Corrected Fourier Series Methods
Malaysia Joural of Mahemaical Scieces 7(): 73-8 (13) MALAYSIAN JOURNAL OF MATHEMATICAL SCIENCES Joural homepage: hp://eispem.upm.edu.my/oural Compariso bewee Fourier ad Correced Fourier Series Mehods 1
More informationSTK4080/9080 Survival and event history analysis
STK48/98 Survival ad eve hisory aalysis Marigales i discree ime Cosider a sochasic process The process M is a marigale if Lecure 3: Marigales ad oher sochasic processes i discree ime (recap) where (formally
More informationλiv Av = 0 or ( λi Av ) = 0. In order for a vector v to be an eigenvector, it must be in the kernel of λi
Liear lgebra Lecure #9 Noes This week s lecure focuses o wha migh be called he srucural aalysis of liear rasformaios Wha are he irisic properies of a liear rasformaio? re here ay fixed direcios? The discussio
More informationODEs II, Supplement to Lectures 6 & 7: The Jordan Normal Form: Solving Autonomous, Homogeneous Linear Systems. April 2, 2003
ODEs II, Suppleme o Lecures 6 & 7: The Jorda Normal Form: Solvig Auoomous, Homogeeous Liear Sysems April 2, 23 I his oe, we describe he Jorda ormal form of a marix ad use i o solve a geeral homogeeous
More informationBig O Notation for Time Complexity of Algorithms
BRONX COMMUNITY COLLEGE of he Ciy Uiversiy of New York DEPARTMENT OF MATHEMATICS AND COMPUTER SCIENCE CSI 33 Secio E01 Hadou 1 Fall 2014 Sepember 3, 2014 Big O Noaio for Time Complexiy of Algorihms Time
More informationAdditional Tables of Simulation Results
Saisica Siica: Suppleme REGULARIZING LASSO: A CONSISTENT VARIABLE SELECTION METHOD Quefeg Li ad Ju Shao Uiversiy of Wiscosi, Madiso, Eas Chia Normal Uiversiy ad Uiversiy of Wiscosi, Madiso Supplemeary
More informationLecture 15 First Properties of the Brownian Motion
Lecure 15: Firs Properies 1 of 8 Course: Theory of Probabiliy II Term: Sprig 2015 Isrucor: Gorda Zikovic Lecure 15 Firs Properies of he Browia Moio This lecure deals wih some of he more immediae properies
More informationIdeal Amplifier/Attenuator. Memoryless. where k is some real constant. Integrator. System with memory
Liear Time-Ivaria Sysems (LTI Sysems) Oulie Basic Sysem Properies Memoryless ad sysems wih memory (saic or dyamic) Causal ad o-causal sysems (Causaliy) Liear ad o-liear sysems (Lieariy) Sable ad o-sable
More informationAvailable online at J. Math. Comput. Sci. 4 (2014), No. 4, ISSN:
Available olie a hp://sci.org J. Mah. Compu. Sci. 4 (2014), No. 4, 716-727 ISSN: 1927-5307 ON ITERATIVE TECHNIQUES FOR NUMERICAL SOLUTIONS OF LINEAR AND NONLINEAR DIFFERENTIAL EQUATIONS S.O. EDEKI *, A.A.
More informationFIXED FUZZY POINT THEOREMS IN FUZZY METRIC SPACE
Mohia & Samaa, Vol. 1, No. II, December, 016, pp 34-49. ORIGINAL RESEARCH ARTICLE OPEN ACCESS FIED FUZZY POINT THEOREMS IN FUZZY METRIC SPACE 1 Mohia S. *, Samaa T. K. 1 Deparme of Mahemaics, Sudhir Memorial
More informationReview Answers for E&CE 700T02
Review Aswers for E&CE 700T0 . Deermie he curre soluio, all possible direcios, ad sepsizes wheher improvig or o for he simple able below: 4 b ma c 0 0 0-4 6 0 - B N B N ^0 0 0 curre sol =, = Ch for - -
More information10.3 Autocorrelation Function of Ergodic RP 10.4 Power Spectral Density of Ergodic RP 10.5 Normal RP (Gaussian RP)
ENGG450 Probabiliy ad Saisics for Egieers Iroducio 3 Probabiliy 4 Probabiliy disribuios 5 Probabiliy Desiies Orgaizaio ad descripio of daa 6 Samplig disribuios 7 Ifereces cocerig a mea 8 Comparig wo reames
More informationCS623: Introduction to Computing with Neural Nets (lecture-10) Pushpak Bhattacharyya Computer Science and Engineering Department IIT Bombay
CS6: Iroducio o Compuig ih Neural Nes lecure- Pushpak Bhaacharyya Compuer Sciece ad Egieerig Deparme IIT Bombay Tilig Algorihm repea A kid of divide ad coquer sraegy Give he classes i he daa, ru he percepro
More informationFour equations describe the dynamic solution to RBC model. Consumption-leisure efficiency condition. Consumption-investment efficiency condition
LINEAR APPROXIMATION OF THE BASELINE RBC MODEL FEBRUARY, 202 Iroducio For f(, y, z ), mulivariable Taylor liear epasio aroud (, yz, ) f (, y, z) f(, y, z) + f (, y, z)( ) + f (, y, z)( y y) + f (, y, z)(
More informationBEST LINEAR FORECASTS VS. BEST POSSIBLE FORECASTS
BEST LINEAR FORECASTS VS. BEST POSSIBLE FORECASTS Opimal ear Forecasig Alhough we have o meioed hem explicily so far i he course, here are geeral saisical priciples for derivig he bes liear forecas, ad
More informationNEWTON METHOD FOR DETERMINING THE OPTIMAL REPLENISHMENT POLICY FOR EPQ MODEL WITH PRESENT VALUE
Yugoslav Joural of Operaios Research 8 (2008, Number, 53-6 DOI: 02298/YUJOR080053W NEWTON METHOD FOR DETERMINING THE OPTIMAL REPLENISHMENT POLICY FOR EPQ MODEL WITH PRESENT VALUE Jeff Kuo-Jug WU, Hsui-Li
More informationLecture 9: Polynomial Approximations
CS 70: Complexiy Theory /6/009 Lecure 9: Polyomial Approximaios Isrucor: Dieer va Melkebeek Scribe: Phil Rydzewski & Piramaayagam Arumuga Naiar Las ime, we proved ha o cosa deph circui ca evaluae he pariy
More informationThe Solution of the One Species Lotka-Volterra Equation Using Variational Iteration Method ABSTRACT INTRODUCTION
Malaysia Joural of Mahemaical Scieces 2(2): 55-6 (28) The Soluio of he Oe Species Loka-Volerra Equaio Usig Variaioal Ieraio Mehod B. Baiha, M.S.M. Noorai, I. Hashim School of Mahemaical Scieces, Uiversii
More informationOnline Supplement to Reactive Tabu Search in a Team-Learning Problem
Olie Suppleme o Reacive abu Search i a eam-learig Problem Yueli She School of Ieraioal Busiess Admiisraio, Shaghai Uiversiy of Fiace ad Ecoomics, Shaghai 00433, People s Republic of Chia, she.yueli@mail.shufe.edu.c
More informationState and Parameter Estimation of The Lorenz System In Existence of Colored Noise
Sae ad Parameer Esimaio of he Lorez Sysem I Eisece of Colored Noise Mozhga Mombeii a Hamid Khaloozadeh b a Elecrical Corol ad Sysem Egieerig Researcher of Isiue for Research i Fudameal Scieces (IPM ehra
More informationOutline. simplest HMM (1) simple HMMs? simplest HMM (2) Parameter estimation for discrete hidden Markov models
Oulie Parameer esimaio for discree idde Markov models Juko Murakami () ad Tomas Taylor (2). Vicoria Uiversiy of Welligo 2. Arizoa Sae Uiversiy Descripio of simple idde Markov models Maximum likeliood esimae
More informationMean Square Convergent Finite Difference Scheme for Stochastic Parabolic PDEs
America Joural of Compuaioal Mahemaics, 04, 4, 80-88 Published Olie Sepember 04 i SciRes. hp://www.scirp.org/joural/ajcm hp://dx.doi.org/0.436/ajcm.04.4404 Mea Square Coverge Fiie Differece Scheme for
More informationSUMMATION OF INFINITE SERIES REVISITED
SUMMATION OF INFINITE SERIES REVISITED I several aricles over he las decade o his web page we have show how o sum cerai iiie series icludig he geomeric series. We wa here o eed his discussio o he geeral
More informationOrder Determination for Multivariate Autoregressive Processes Using Resampling Methods
joural of mulivariae aalysis 57, 175190 (1996) aricle o. 0028 Order Deermiaio for Mulivariae Auoregressive Processes Usig Resamplig Mehods Chaghua Che ad Richard A. Davis* Colorado Sae Uiversiy ad Peer
More informationA TAUBERIAN THEOREM FOR THE WEIGHTED MEAN METHOD OF SUMMABILITY
U.P.B. Sci. Bull., Series A, Vol. 78, Iss. 2, 206 ISSN 223-7027 A TAUBERIAN THEOREM FOR THE WEIGHTED MEAN METHOD OF SUMMABILITY İbrahim Çaak I his paper we obai a Tauberia codiio i erms of he weighed classical
More informationSIGNAL estimation has been a fundamental problem in a
90 IEEE TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING OVER NETWORKS, VOL., NO. 3, SEPTEMBER 06 A Olie Parallel Algorihm for Recursive Esimaio of Sparse Sigals Yag Yag, Member, IEEE, Marius Pesaveo,
More informationThe analysis of the method on the one variable function s limit Ke Wu
Ieraioal Coferece o Advaces i Mechaical Egieerig ad Idusrial Iformaics (AMEII 5) The aalysis of he mehod o he oe variable fucio s i Ke Wu Deparme of Mahemaics ad Saisics Zaozhuag Uiversiy Zaozhuag 776
More informationSome Properties of Semi-E-Convex Function and Semi-E-Convex Programming*
The Eighh Ieraioal Symposium o Operaios esearch ad Is Applicaios (ISOA 9) Zhagjiajie Chia Sepember 2 22 29 Copyrigh 29 OSC & APOC pp 33 39 Some Properies of Semi-E-Covex Fucio ad Semi-E-Covex Programmig*
More informationExtremal graph theory II: K t and K t,t
Exremal graph heory II: K ad K, Lecure Graph Theory 06 EPFL Frak de Zeeuw I his lecure, we geeralize he wo mai heorems from he las lecure, from riagles K 3 o complee graphs K, ad from squares K, o complee
More informationNotes 03 largely plagiarized by %khc
1 1 Discree-Time Covoluio Noes 03 largely plagiarized by %khc Le s begi our discussio of covoluio i discree-ime, sice life is somewha easier i ha domai. We sar wih a sigal x[] ha will be he ipu io our
More informationMath 6710, Fall 2016 Final Exam Solutions
Mah 67, Fall 6 Fial Exam Soluios. Firs, a sude poied ou a suble hig: if P (X i p >, he X + + X (X + + X / ( evaluaes o / wih probabiliy p >. This is roublesome because a radom variable is supposed o be
More informationF D D D D F. smoothed value of the data including Y t the most recent data.
Module 2 Forecasig 1. Wha is forecasig? Forecasig is defied as esimaig he fuure value ha a parameer will ake. Mos scieific forecasig mehods forecas he fuure value usig pas daa. I Operaios Maageme forecasig
More informationActuarial Society of India
Acuarial Sociey of Idia EXAMINAIONS Jue 5 C4 (3) Models oal Marks - 5 Idicaive Soluio Q. (i) a) Le U deoe he process described by 3 ad V deoe he process described by 4. he 5 e 5 PU [ ] PV [ ] ( e ).538!
More informationReview Exercises for Chapter 9
0_090R.qd //0 : PM Page 88 88 CHAPTER 9 Ifiie Series I Eercises ad, wrie a epressio for he h erm of he sequece..,., 5, 0,,,, 0,... 7,... I Eercises, mach he sequece wih is graph. [The graphs are labeled
More informationA Complex Neural Network Algorithm for Computing the Largest Real Part Eigenvalue and the corresponding Eigenvector of a Real Matrix
4h Ieraioal Coferece o Sesors, Mecharoics ad Auomaio (ICSMA 06) A Complex Neural Newor Algorihm for Compuig he Larges eal Par Eigevalue ad he correspodig Eigevecor of a eal Marix HANG AN, a, XUESONG LIANG,
More information1. Solve by the method of undetermined coefficients and by the method of variation of parameters. (4)
7 Differeial equaios Review Solve by he mehod of udeermied coefficies ad by he mehod of variaio of parameers (4) y y = si Soluio; we firs solve he homogeeous equaio (4) y y = 4 The correspodig characerisic
More information11. Adaptive Control in the Presence of Bounded Disturbances Consider MIMO systems in the form,
Lecure 6. Adapive Corol i he Presece of Bouded Disurbaces Cosider MIMO sysems i he form, x Aref xbu x Bref ycmd (.) y Cref x operaig i he presece of a bouded ime-depede disurbace R. All he assumpios ad
More informationEconomics 8723 Macroeconomic Theory Problem Set 2 Professor Sanjay Chugh Spring 2017
Deparme of Ecoomics The Ohio Sae Uiversiy Ecoomics 8723 Macroecoomic Theory Problem Se 2 Professor Sajay Chugh Sprig 207 Labor Icome Taxes, Nash-Bargaied Wages, ad Proporioally-Bargaied Wages. I a ecoomy
More informationSection 8 Convolution and Deconvolution
APPLICATIONS IN SIGNAL PROCESSING Secio 8 Covoluio ad Decovoluio This docume illusraes several echiques for carryig ou covoluio ad decovoluio i Mahcad. There are several operaors available for hese fucios:
More informationSolution. 1 Solutions of Homework 6. Sangchul Lee. April 28, Problem 1.1 [Dur10, Exercise ]
Soluio Sagchul Lee April 28, 28 Soluios of Homework 6 Problem. [Dur, Exercise 2.3.2] Le A be a sequece of idepede eves wih PA < for all. Show ha P A = implies PA i.o. =. Proof. Noice ha = P A c = P A c
More informationIf boundary values are necessary, they are called mixed initial-boundary value problems. Again, the simplest prototypes of these IV problems are:
3. Iiial value problems: umerical soluio Fiie differeces - Trucaio errors, cosisecy, sabiliy ad covergece Crieria for compuaioal sabiliy Explici ad implici ime schemes Table of ime schemes Hyperbolic ad
More informationAsymptotic statistics for multilayer perceptron with ReLu hidden units
ESANN 8 proceedigs, Europea Symposium o Arificial Neural Neworks, Compuaioal Ielligece ad Machie Learig. Bruges (Belgium), 5-7 April 8, i6doc.com publ., ISBN 978-8758747-6. Available from hp://www.i6doc.com/e/.
More informationCOS 522: Complexity Theory : Boaz Barak Handout 10: Parallel Repetition Lemma
COS 522: Complexiy Theory : Boaz Barak Hadou 0: Parallel Repeiio Lemma Readig: () A Parallel Repeiio Theorem / Ra Raz (available o his websie) (2) Parallel Repeiio: Simplificaios ad he No-Sigallig Case
More informationCalculus BC 2015 Scoring Guidelines
AP Calculus BC 5 Scorig Guidelies 5 The College Board. College Board, Advaced Placeme Program, AP, AP Ceral, ad he acor logo are regisered rademarks of he College Board. AP Ceral is he official olie home
More informationA Generalized Cost Malmquist Index to the Productivities of Units with Negative Data in DEA
Proceedigs of he 202 Ieraioal Coferece o Idusrial Egieerig ad Operaios Maageme Isabul, urey, July 3 6, 202 A eeralized Cos Malmquis Ide o he Produciviies of Uis wih Negaive Daa i DEA Shabam Razavya Deparme
More informationA note on deviation inequalities on {0, 1} n. by Julio Bernués*
A oe o deviaio iequaliies o {0, 1}. by Julio Berués* Deparameo de Maemáicas. Faculad de Ciecias Uiversidad de Zaragoza 50009-Zaragoza (Spai) I. Iroducio. Le f: (Ω, Σ, ) IR be a radom variable. Roughly
More informationCalculus Limits. Limit of a function.. 1. One-Sided Limits...1. Infinite limits 2. Vertical Asymptotes...3. Calculating Limits Using the Limit Laws.
Limi of a fucio.. Oe-Sided..... Ifiie limis Verical Asympoes... Calculaig Usig he Limi Laws.5 The Squeeze Theorem.6 The Precise Defiiio of a Limi......7 Coiuiy.8 Iermediae Value Theorem..9 Refereces..
More informationANALYSIS OF THE CHAOS DYNAMICS IN (X n,x n+1) PLANE
ANALYSIS OF THE CHAOS DYNAMICS IN (X,X ) PLANE Soegiao Soelisioo, The Houw Liog Badug Isiue of Techolog (ITB) Idoesia soegiao@sude.fi.ib.ac.id Absrac I he las decade, sudies of chaoic ssem are more ofe
More informationAn Online Parallel and Distributed Algorithm for Recursive Estimation of Sparse Signals
A Olie Parallel ad Disribued Algorihm for Recursive simaio of Sparse Sigals Yag Yag, Megyi Zhag, Marius Pesaveo, ad Daiel P. Palomar Absrac I his paper, we cosider a recursive esimaio problem for liear
More informationApproximating Solutions for Ginzburg Landau Equation by HPM and ADM
Available a hp://pvamu.edu/aam Appl. Appl. Mah. ISSN: 193-9466 Vol. 5, No. Issue (December 1), pp. 575 584 (Previously, Vol. 5, Issue 1, pp. 167 1681) Applicaios ad Applied Mahemaics: A Ieraioal Joural
More informationINTEGER INTERVAL VALUE OF NEWTON DIVIDED DIFFERENCE AND FORWARD AND BACKWARD INTERPOLATION FORMULA
Volume 8 No. 8, 45-54 ISSN: 34-3395 (o-lie versio) url: hp://www.ijpam.eu ijpam.eu INTEGER INTERVAL VALUE OF NEWTON DIVIDED DIFFERENCE AND FORWARD AND BACKWARD INTERPOLATION FORMULA A.Arul dass M.Dhaapal
More informationMathematical Statistics. 1 Introduction to the materials to be covered in this course
Mahemaical Saisics Iroducio o he maerials o be covered i his course. Uivariae & Mulivariae r.v s 2. Borl-Caelli Lemma Large Deviaios. e.g. X,, X are iid r.v s, P ( X + + X where I(A) is a umber depedig
More informationLecture 8 April 18, 2018
Sas 300C: Theory of Saisics Sprig 2018 Lecure 8 April 18, 2018 Prof Emmauel Cades Scribe: Emmauel Cades Oulie Ageda: Muliple Tesig Problems 1 Empirical Process Viewpoi of BHq 2 Empirical Process Viewpoi
More informationEnhanced Online Subspace Estimation via Adaptive Sensing
Ehaced Olie Subspace Esimaio via Adapive Sesig Greg Ogie, David Hog, Dejiao Zhag, Laura Balzao Deparme of Elecrical Egieerig ad Compuer Sciece Uiversiy of Michiga A Arbor, MI 488 e-mail: {gogie,dahog,dejiao,girasole}@umich.edu
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 4 9/16/2013. Applications of the large deviation technique
MASSACHUSETTS ISTITUTE OF TECHOLOGY 6.265/5.070J Fall 203 Lecure 4 9/6/203 Applicaios of he large deviaio echique Coe.. Isurace problem 2. Queueig problem 3. Buffer overflow probabiliy Safey capial for
More informationProcedia - Social and Behavioral Sciences 230 ( 2016 ) Joint Probability Distribution and the Minimum of a Set of Normalized Random Variables
Available olie a wwwsciecedireccom ScieceDirec Procedia - Social ad Behavioral Scieces 30 ( 016 ) 35 39 3 rd Ieraioal Coferece o New Challeges i Maageme ad Orgaizaio: Orgaizaio ad Leadership, May 016,
More informationDavid Randall. ( )e ikx. k = u x,t. u( x,t)e ikx dx L. x L /2. Recall that the proof of (1) and (2) involves use of the orthogonality condition.
! Revised April 21, 2010 1:27 P! 1 Fourier Series David Radall Assume ha u( x,) is real ad iegrable If he domai is periodic, wih period L, we ca express u( x,) exacly by a Fourier series expasio: ( ) =
More informationTAKA KUSANO. laculty of Science Hrosh tlnlersty 1982) (n-l) + + Pn(t)x 0, (n-l) + + Pn(t)Y f(t,y), XR R are continuous functions.
Iera. J. Mah. & Mah. Si. Vol. 6 No. 3 (1983) 559-566 559 ASYMPTOTIC RELATIOHIPS BETWEEN TWO HIGHER ORDER ORDINARY DIFFERENTIAL EQUATIONS TAKA KUSANO laculy of Sciece Hrosh llersy 1982) ABSTRACT. Some asympoic
More informationDynamic h-index: the Hirsch index in function of time
Dyamic h-idex: he Hirsch idex i fucio of ime by L. Egghe Uiversiei Hassel (UHassel), Campus Diepebeek, Agoralaa, B-3590 Diepebeek, Belgium ad Uiversiei Awerpe (UA), Campus Drie Eike, Uiversieisplei, B-260
More informationInference of the Second Order Autoregressive. Model with Unit Roots
Ieraioal Mahemaical Forum Vol. 6 0 o. 5 595-604 Iferece of he Secod Order Auoregressive Model wih Ui Roos Ahmed H. Youssef Professor of Applied Saisics ad Ecoomerics Isiue of Saisical Sudies ad Research
More informationAdaBoost. AdaBoost: Introduction
Slides modified from: MLSS 03: Guar Räsch, Iroducio o Boosig hp://www.boosig.org : Iroducio 2 Classifiers Supervised Classifiers Liear Classifiers Percepro, Leas Squares Mehods Liear SVM Noliear Classifiers
More informationLINEAR APPROXIMATION OF THE BASELINE RBC MODEL JANUARY 29, 2013
LINEAR APPROXIMATION OF THE BASELINE RBC MODEL JANUARY 29, 203 Iroducio LINEARIZATION OF THE RBC MODEL For f( x, y, z ) = 0, mulivariable Taylor liear expasio aroud f( x, y, z) f( x, y, z) + f ( x, y,
More informationResearch Article A Generalized Nonlinear Sum-Difference Inequality of Product Form
Joural of Applied Mahemaics Volume 03, Aricle ID 47585, 7 pages hp://dx.doi.org/0.55/03/47585 Research Aricle A Geeralized Noliear Sum-Differece Iequaliy of Produc Form YogZhou Qi ad Wu-Sheg Wag School
More information2 f(x) dx = 1, 0. 2f(x 1) dx d) 1 4t t6 t. t 2 dt i)
Mah PracTes Be sure o review Lab (ad all labs) There are los of good quesios o i a) Sae he Mea Value Theorem ad draw a graph ha illusraes b) Name a impora heorem where he Mea Value Theorem was used i he
More informationLINEAR APPROXIMATION OF THE BASELINE RBC MODEL SEPTEMBER 17, 2013
LINEAR APPROXIMATION OF THE BASELINE RBC MODEL SEPTEMBER 7, 203 Iroducio LINEARIZATION OF THE RBC MODEL For f( xyz,, ) = 0, mulivariable Taylor liear expasio aroud f( xyz,, ) f( xyz,, ) + f( xyz,, )( x
More informationBE.430 Tutorial: Linear Operator Theory and Eigenfunction Expansion
BE.43 Tuorial: Liear Operaor Theory ad Eigefucio Expasio (adaped fro Douglas Lauffeburger) 9//4 Moivaig proble I class, we ecouered parial differeial equaios describig rasie syses wih cheical diffusio.
More informationFour equations describe the dynamic solution to RBC model. Consumption-leisure efficiency condition. Consumption-investment efficiency condition
LINEARIZING AND APPROXIMATING THE RBC MODEL SEPTEMBER 7, 200 For f( x, y, z ), mulivariable Taylor liear expasio aroud ( x, yz, ) f ( x, y, z) f( x, y, z) + f ( x, y, z)( x x) + f ( x, y, z)( y y) + f
More informationAn interesting result about subset sums. Nitu Kitchloo. Lior Pachter. November 27, Abstract
A ieresig resul abou subse sums Niu Kichloo Lior Pacher November 27, 1993 Absrac We cosider he problem of deermiig he umber of subses B f1; 2; : : :; g such ha P b2b b k mod, where k is a residue class
More information12 Getting Started With Fourier Analysis
Commuicaios Egieerig MSc - Prelimiary Readig Geig Sared Wih Fourier Aalysis Fourier aalysis is cocered wih he represeaio of sigals i erms of he sums of sie, cosie or complex oscillaio waveforms. We ll
More informationO & M Cost O & M Cost
5/5/008 Turbie Reliabiliy, Maieace ad Faul Deecio Zhe Sog, Adrew Kusiak 39 Seamas Ceer Iowa Ciy, Iowa 54-57 adrew-kusiak@uiowa.edu Tel: 39-335-5934 Fax: 39-335-5669 hp://www.icae.uiowa.edu/~akusiak Oulie
More informationCSE 241 Algorithms and Data Structures 10/14/2015. Skip Lists
CSE 41 Algorihms ad Daa Srucures 10/14/015 Skip Liss This hadou gives he skip lis mehods ha we discussed i class. A skip lis is a ordered, doublyliked lis wih some exra poiers ha allow us o jump over muliple
More informationMoment Generating Function
1 Mome Geeraig Fucio m h mome m m m E[ ] x f ( x) dx m h ceral mome m m m E[( ) ] ( ) ( x ) f ( x) dx Mome Geeraig Fucio For a real, M () E[ e ] e k x k e p ( x ) discree x k e f ( x) dx coiuous Example
More informationF.Y. Diploma : Sem. II [AE/CH/FG/ME/PT/PG] Applied Mathematics
F.Y. Diploma : Sem. II [AE/CH/FG/ME/PT/PG] Applied Mahemaics Prelim Quesio Paper Soluio Q. Aemp ay FIVE of he followig : [0] Q.(a) Defie Eve ad odd fucios. [] As.: A fucio f() is said o be eve fucio if
More informationMATH 507a ASSIGNMENT 4 SOLUTIONS FALL 2018 Prof. Alexander. g (x) dx = g(b) g(0) = g(b),
MATH 57a ASSIGNMENT 4 SOLUTIONS FALL 28 Prof. Alexader (2.3.8)(a) Le g(x) = x/( + x) for x. The g (x) = /( + x) 2 is decreasig, so for a, b, g(a + b) g(a) = a+b a g (x) dx b so g(a + b) g(a) + g(b). Sice
More informationThe Moment Approximation of the First Passage Time for the Birth Death Diffusion Process with Immigraton to a Moving Linear Barrier
America Joural of Applied Mahemaics ad Saisics, 015, Vol. 3, No. 5, 184-189 Available olie a hp://pubs.sciepub.com/ajams/3/5/ Sciece ad Educaio Publishig DOI:10.1691/ajams-3-5- The Mome Approximaio of
More informationElectrical Engineering Department Network Lab.
Par:- Elecrical Egieerig Deparme Nework Lab. Deermiaio of differe parameers of -por eworks ad verificaio of heir ierrelaio ships. Objecive: - To deermie Y, ad ABD parameers of sigle ad cascaded wo Por
More informationSolutions to Problems 3, Level 4
Soluios o Problems 3, Level 4 23 Improve he resul of Quesio 3 whe l. i Use log log o prove ha for real >, log ( {}log + 2 d log+ P ( + P ( d 2. Here P ( is defied i Quesio, ad parial iegraio has bee used.
More informationB. Maddah INDE 504 Simulation 09/02/17
B. Maddah INDE 54 Simulaio 9/2/7 Queueig Primer Wha is a queueig sysem? A queueig sysem cosiss of servers (resources) ha provide service o cusomers (eiies). A Cusomer requesig service will sar service
More informationA Note on Random k-sat for Moderately Growing k
A Noe o Radom k-sat for Moderaely Growig k Ju Liu LMIB ad School of Mahemaics ad Sysems Sciece, Beihag Uiversiy, Beijig, 100191, P.R. Chia juliu@smss.buaa.edu.c Zogsheg Gao LMIB ad School of Mahemaics
More informationSolutions to selected problems from the midterm exam Math 222 Winter 2015
Soluios o seleced problems from he miderm eam Mah Wier 5. Derive he Maclauri series for he followig fucios. (cf. Pracice Problem 4 log( + (a L( d. Soluio: We have he Maclauri series log( + + 3 3 4 4 +...,
More informationC(p, ) 13 N. Nuclear reactions generate energy create new isotopes and elements. Notation for stellar rates: p 12
Iroducio o sellar reacio raes Nuclear reacios geerae eergy creae ew isoopes ad elemes Noaio for sellar raes: p C 3 N C(p,) 3 N The heavier arge ucleus (Lab: arge) he ligher icomig projecile (Lab: beam)
More informationVIM for Determining Unknown Source Parameter in Parabolic Equations
ISSN 1746-7659, Eglad, UK Joural of Iformaio ad Compuig Sciece Vol. 11, No., 16, pp. 93-1 VIM for Deermiig Uko Source Parameer i Parabolic Equaios V. Eskadari *ad M. Hedavad Educaio ad Traiig, Dourod,
More informationPage 1. Before-After Control-Impact (BACI) Power Analysis For Several Related Populations. Richard A. Hinrichsen. March 3, 2010
Page Before-Afer Corol-Impac BACI Power Aalysis For Several Relaed Populaios Richard A. Hirichse March 3, Cavea: This eperimeal desig ool is for a idealized power aalysis buil upo several simplifyig assumpios
More informationA Note on Prediction with Misspecified Models
ITB J. Sci., Vol. 44 A, No. 3,, 7-9 7 A Noe o Predicio wih Misspecified Models Khresha Syuhada Saisics Research Divisio, Faculy of Mahemaics ad Naural Scieces, Isiu Tekologi Badug, Jala Gaesa Badug, Jawa
More informationCLOSED FORM EVALUATION OF RESTRICTED SUMS CONTAINING SQUARES OF FIBONOMIAL COEFFICIENTS
PB Sci Bull, Series A, Vol 78, Iss 4, 2016 ISSN 1223-7027 CLOSED FORM EVALATION OF RESTRICTED SMS CONTAINING SQARES OF FIBONOMIAL COEFFICIENTS Emrah Kılıc 1, Helmu Prodiger 2 We give a sysemaic approach
More informationFresnel Dragging Explained
Fresel Draggig Explaied 07/05/008 Decla Traill Decla@espace.e.au The Fresel Draggig Coefficie required o explai he resul of he Fizeau experime ca be easily explaied by usig he priciples of Eergy Field
More informationA Novel Approach for Solving Burger s Equation
Available a hp://pvamu.edu/aam Appl. Appl. Mah. ISSN: 93-9466 Vol. 9, Issue (December 4), pp. 54-55 Applicaios ad Applied Mahemaics: A Ieraioal Joural (AAM) A Novel Approach for Solvig Burger s Equaio
More informationLecture 15: Three-tank Mixing and Lead Poisoning
Lecure 15: Three-ak Miig ad Lead Poisoig Eigevalues ad eigevecors will be used o fid he soluio of a sysem for ukow fucios ha saisfy differeial equaios The ukow fucios will be wrie as a 1 colum vecor [
More informationUsing Linnik's Identity to Approximate the Prime Counting Function with the Logarithmic Integral
Usig Lii's Ideiy o Approimae he Prime Couig Fucio wih he Logarihmic Iegral Naha McKezie /26/2 aha@icecreambreafas.com Summary:This paper will show ha summig Lii's ideiy from 2 o ad arragig erms i a cerai
More informationProblems and Solutions for Section 3.2 (3.15 through 3.25)
3-7 Problems ad Soluios for Secio 3 35 hrough 35 35 Calculae he respose of a overdamped sigle-degree-of-freedom sysem o a arbirary o-periodic exciaio Soluio: From Equaio 3: x = # F! h "! d! For a overdamped
More informationK3 p K2 p Kp 0 p 2 p 3 p
Mah 80-00 Mo Ar 0 Chaer 9 Fourier Series ad alicaios o differeial equaios (ad arial differeial equaios) 9.-9. Fourier series defiiio ad covergece. The idea of Fourier series is relaed o he liear algebra
More informationBasic Results in Functional Analysis
Preared by: F.. ewis Udaed: Suday, Augus 7, 4 Basic Resuls i Fucioal Aalysis f ( ): X Y is coiuous o X if X, (, ) z f( z) f( ) f ( ): X Y is uiformly coiuous o X if i is coiuous ad ( ) does o deed o. f
More informationClock Skew and Signal Representation
Clock Skew ad Sigal Represeaio Ch. 7 IBM Power 4 Chip 0/7/004 08 frequecy domai Program Iroducio ad moivaio Sequeial circuis, clock imig, Basic ools for frequecy domai aalysis Fourier series sigal represeaio
More informationApproximately Quasi Inner Generalized Dynamics on Modules. { } t t R
Joural of Scieces, Islamic epublic of Ira 23(3): 245-25 (22) Uiversiy of Tehra, ISSN 6-4 hp://jscieces.u.ac.ir Approximaely Quasi Ier Geeralized Dyamics o Modules M. Mosadeq, M. Hassai, ad A. Nikam Deparme
More informationL-functions and Class Numbers
L-fucios ad Class Numbers Sude Number Theory Semiar S. M.-C. 4 Sepember 05 We follow Romyar Sharifi s Noes o Iwasawa Theory, wih some help from Neukirch s Algebraic Number Theory. L-fucios of Dirichle
More informationStochastic Processes Adopted From p Chapter 9 Probability, Random Variables and Stochastic Processes, 4th Edition A. Papoulis and S.
Sochasic Processes Adoped From p Chaper 9 Probabiliy, adom Variables ad Sochasic Processes, 4h Ediio A. Papoulis ad S. Pillai 9. Sochasic Processes Iroducio Le deoe he radom oucome of a experime. To every
More informationPrinciples of Communications Lecture 1: Signals and Systems. Chih-Wei Liu 劉志尉 National Chiao Tung University
Priciples of Commuicaios Lecure : Sigals ad Sysems Chih-Wei Liu 劉志尉 Naioal Chiao ug Uiversiy cwliu@wis.ee.cu.edu.w Oulies Sigal Models & Classificaios Sigal Space & Orhogoal Basis Fourier Series &rasform
More informationChapter 2: Time-Domain Representations of Linear Time-Invariant Systems. Chih-Wei Liu
Caper : Time-Domai Represeaios of Liear Time-Ivaria Sysems Ci-Wei Liu Oulie Iroucio Te Covoluio Sum Covoluio Sum Evaluaio Proceure Te Covoluio Iegral Covoluio Iegral Evaluaio Proceure Iercoecios of LTI
More information