arxiv: v1 [math.oc] 11 Dec 2014

Size: px
Start display at page:

Download "arxiv: v1 [math.oc] 11 Dec 2014"

Transcription

1 Nework Newon Aryan Mokhar, Qng Lng and Alejandro Rbero Dep. of Elecrcal and Sysems Engneerng, Unversy of Pennsylvana Dep. of Auomaon, Unversy of Scence and Technology of Chna arxv: v1 [mah.oc] 11 Dec 214 Absrac We consder mnmzaon of a sum of conve objecve funcons where he componens of he objecve are avalable a dfferen nodes of a nework and nodes are allowed o only communcae wh her neghbors. The use of dsrbued subgraden or graden mehods s wdespread bu hey ofen suffer from slow convergence snce hey rely on frs order nformaon, whch leads o a large number of local communcaons beween nodes n he nework. In hs paper we propose he Nework Newon NN) mehod as a dsrbued algorhm ha ncorporaes second order nformaon va dsrbued evaluaon of appromaons o Newon seps. We also nroduce adapve A)NN n order o esablsh eac convergence. Numercal analyses show sgnfcan mprovemen n boh convergence me and number of communcaons for NN relave o esng frs order) alernaves. I. INTRODUCTION Dsrbued opmzaon algorhms are used o mnmze a global cos funcon over a se of nodes n suaons where he objecve funcon s defned as a sum of a se of local funcons. Consder a varable R p and a conneced nework conanng n agens each of whch has access o a local funcon f : R p R. The agens cooperae n mnmzng he aggregae cos funcon f : R p R akng values f) := f). I.e., agens cooperae n solvng he global opmzaon problem := argmn f) = argmn f ). 1) Problems of hs form arse ofen n, e.g., wreless sysems [1], [2], sensor neworks [3], [4], and large scale machne learnng [5]. There are dfferen algorhms o solve 1) n a dsrbued manner. The mos popular alernaves are decenralzed graden descen ) [6] [9], dsrbued mplemenaons of he alernang drecon mehod of mulplers [3], [1] [12], and decenralzed dual averagng DDA) [13]. A feaure common o all of hese algorhms s he slow convergence rae n llcondoned problems snce hey operae on frs order nformaon only. Ths paper consders Nework Newon NN), a mehod ha reles on dsrbued appromaons of Newon seps for he global cos funcon f o accelerae convergence of he algorhm. We begn hs paper by nroducng he dea ha solves a penalzed verson of 1) usng graden descen n leu of solvng he orgnal opmzaon problem. To accelerae he convergence of graden descen mehod for solvng he penaly verson of 1) we advocae he use of he NN algorhm. Ths algorhm reles on appromaons o he Newon sep of he penalzed objecve funcon by runcang he Taylor seres of he eac Newon sep Secon II-A). These appromaons o he Newon sep can be compued n a dsrbued manner wh a level of localy conrolled by he number K of elemens ha are reaned n he Taylor s seres. When we rean K elemens n he seres we say ha we mplemen NN-K. We prove ha for a fed penaly coeffcen lower and upper bounds on he Hessans of local objecve funcons f are suffcen o guaranee a leas lnear convergence of NN-K o he opmal argumens of penalzed opmzaon problem Theorem 1). Furher, We nroduce an adapve verson of NN-K ANN-K) ha uses an ncreasng penaly coeffcen o acheve eac convergence o he opmal soluon of 1) Secon II-B). We sudy he advanages of NN-K relave o, boh n erms of number of eraons and communcaons for convergence for solvng a famly of quadrac objecve problems Secon IV). Work n hs paper s suppored by ARO W911NF , NSF CAREER CCF , and ONR N II. PROBLEM FORMULATION AND ALGORITHM DEFINITION The nework ha connecs he agens s assumed symmerc and specfed by he neghborhoods N ha conan he ls of nodes han can communcae wh for = 1,..., n. s an esablshed dsrbued mehod o solve 1) whch reles on he nroducon of local varables R p and nonnegave weghs w j ha are no null f and only f j = or f j N. Leng N be a dscree me nde and α a gven sepsze, s defned by he recurson,+1 = w j j, α f,), = 1,..., n. 2) j=1 Snce w j = when j and j / N, follows from 2) ha each agen updaes s esmae of he opmal vecor by performng an average over he esmaes j, of s neghbors j N and s own esmae,, and descendng hrough he negave local graden f,). Noe ha weghs w j ha nodes assgn o each oher form a wegh mar W R n n ha s symmerc and row sochasc. I s also cusomary o requre he rank of I W o be n 1 so ha nulli W) = span1). If he wo assumpons W T = W and nulli W) = 1 are rue, s possble o show ha 2) approaches he soluon of 1) n he sense ha, for all and large, [6]. To rewre 2) defne he mar Z := W I R np np as he Kronecker produc of wegh mar W R n n and he deny mar I R p p. Furher, we nroduce vecors y := [ ] 1;... ; n [ R np ha concaenaes he local vecors, and vecor hy) := f1 1);... ; f ] n n) R np whch concaenaes he gradens of he local funcons f aken wh respec o he local varable. I s hen ready o see ha 2) s equvalen o y +1 = Zy αhy ) = y [ I Z)y + αhy ) ], 3) where n he second equaly we added and subraced y and regrouped erms. Inspecon of 3) reveals ha he updae formula a sep s equvalen o a regular) graden descen algorhm beng used o solve he program y := argmn F y) := mn 1 2 yt I Z) y + α f ). 4) Observe ha s possble o wre he graden of F y) as g := F y ) = I Z)y + αhy ), 5) n order o wre 3) as y +1 = y g and conclude ha descends along he negave graden of F y) wh un sepsze. The epresson n 2) s jus a local mplemenaon of 5) where node mplemens he descen,+1 =, g, where g, s he h elemen of he graden g = [g,;... ; g,]. Node can compue he local graden g, = 1 w ), j N w j j, + α f,). 6) Noce ha snce we know ha he null space of I W s nulli W) = span1) and ha Z = W I, we oban ha he span of I Z s nulli Z) = span1 I). Thus, we have ha I Z)y = holds f and only f 1 = = n. Snce he mar I Z s posve semdefne because s sochasc and symmerc, he same s rue of he square roo mar I Z) 1/2. Therefore, we have ha he opmzaon problem

2 2 n 1) s equvalen o he opmzaon problem ỹ := argmn f ), s.. I Z) 1/2 y =. 7) Indeed, for y = [ 1;... ; n] o be feasble n 7) we mus have 1 = = n because null[i Z) 1/2 ] = span1 I) as already argued. When resrced o hs feasble se he objecve f) of 7) s he same as he objecve of 1) from where follows ha a soluon ỹ = [ 1;... ; n] of 7) s such ha = = for all,.e. ỹ = [ 1;... ; n]. The unconsraned mnmzaon n 4) s a penaly verson of 7). The penaly funcon assocaed wh he consran I Z) 1/2 y = s he squared norm 1/2) I Z) 1/2 y 2 and he correspondng penaly coeffcen s 1/α. Inasmuch as he penaly coeffcen 1/α s suffcenly large, he opmal argumens y and ỹ are no oo far apar. In hs paper we eplo he renerpreaon of 3) as a mehod o mnmze 4) o propose an appromae Newon algorhm ha can be mplemened n a dsrbued manner. We eplan hs algorhm n he followng secon. A. Nework Newon Insead of solvng 4) wh a graden descen algorhm as n, we can solve 4) usng Newon s mehod. To mplemen Newon s mehod we need o compue he Hessan H := 2 F y ) of F evaluaed a y so as o deermne he Newon sep d := H 1 g. Sar by dfferenang wce n 4) n order o wre H as H := 2 F y ) = I Z + αg, 8) where he mar G R np np s a block dagonal mar formed by blocks G, R p p conanng he Hessan of he h local funcon, G, = 2 f,). 9) I follows from 8) and 9) ha he Hessan H s block sparse wh blocks H j, R p p havng he sparsy paern of Z, whch s he sparsy paern of he graph. The dagonal blocks are of he form H, = 1 w )I + α 2 f,) and he off dagonal blocks are no null only when j N n whch case H j, = w ji. Whle he Hessan H s sparse, he nverse H s no. I s he laer ha we need o compue he Newon sep d := H 1 g. To overcome hs problem we spl he dagonal and off-dagonal blocks of H and rely on a Taylor s epanson of he nverse. To be precse, wre H = D B where he mar D s defned as D := αg + 2 I dagz)) := αg + 2 I Z d ), 1) n he second equaly we defned Z d := dagz) for fuure reference. Snce he dagonal weghs mus be w < 1, he mar I Z d s posve defne. The same s rue of he block dagonal mar G because he local funcons are assumed srongly conve. Therefore, he mar D s block dagonal and posve defne. The h dagonal block D, R p of D can be compued and sored by node as D, = α 2 f,) + 21 w )I. To have H = D B we mus defne B := D H. Consderng he defnons of H and D n 8) and 1), follows ha B = I 2Z d + Z. 11) Observe ha B s ndependen of me and depends on he wegh mar Z only. As n he case of he Hessan H, he mar B s block sparse wh wh blocks B j R p p havng he sparsy paern of Z, whch s he sparsy paern of he graph. Node can compue he dagonal blocks B = 1 w )I and he off dagonal blocks B j = w ji usng he local nformaon abou s own weghs only. Proceed now o facor D 1/2 from boh sdes of he splng relaonshp o wre H = D 1/2 I D 1/2 BD 1/2 ) 1 D 1/2. When we consder he Hessan nverse H 1, we can use he Taylor seres I X) 1 = Algorhm 1 Nework Newon-K mehod a node Requre: Inal erae,. 1: for =, 1, 2,... do 2: Echange eraes, wh neghbors j N. 3: Graden: g, = 1 w ), 4: Compue descen drecon d ), 5: for k =,..., K 1 do 6: Echange local elemens d k), 7: NN-k + 1) sep: d k+1), j N w j j, + α f,). = D 1, g, of he NN-k sep wh neghbors B jd k)., j N,j= 8: end for 9: Updae local erae:,+1 =, + ɛ d K),. 1: end for j= Xj wh X /2 B o wre H 1 /2 k= j, g, B 1/2 D. 12) Observe ha he sum n 12) converges f he absolue value of all he egenvalues of mar B are srcly less han 1. Ths resul s proven n [14]. Nework Newon NN) s defned as a famly of algorhms ha rely on runcaons of he seres n 12). The Kh member of hs famly, NN-K consders he frs K + 1 erms of he seres o defne he appromae Hessan nverse Ĥ K) 1 :/2 K k= B 1/2 D. 13) NN-K uses he appromae Hessan ĤK) 1 as a curvaure correcon mar ha s used n leu of he eac Hessan nverse H 1 o esmae he Newon sep. I.e., nsead of descendng along he Newon sep d := H 1 g we descend along he NN-K sep d K) := ĤK) 1 g, whch we nend as an appromaon of d. Usng he eplc epresson for n 13) we wre he NN-K sep as Ĥ K) 1 d K) = K k= B 1/2 D g, 14) where, we recall, he vecor g s he graden of objecve funcon F y) defned n 5). The NN-K updae formula can hen be wren as y +1 = y + ɛ d K). 15) The algorhm defned by recursve applcaon of 15) can be mplemened n a dsrbued manner because he runcaed seres n 13) has a local srucure conrolled by he parameer K. To eplan hs saemen beer defne he componens d K), R p of he NN-K sep d K) = [d K) 1, ;... ; dk) n, ]. A dsrbued mplemenaon of 15) requres ha node compues d K), so as o mplemen he local descen,+1 =, + ɛd K),. The sep componens dk), can be compued hrough local compuaons. To see ha hs s rue frs noe ha consderng he defnon of he NN-K descen drecon n 14) he sequence of NN descen drecons sasfes d k+1) Bd k) d k+1), D 1 g Bd k) g ). 16) Then observe ha snce he mar ˆB has he sparsy paern of he graph, hs recurson can be decomposed no local componens, j N,j= B jd k) j, g, ), 17) The mar D, = α 2 f,)+21 w )I s sored and compued a

3 3 Algorhm 2 Compuaon of NN-K sep a node. 1: funcon = NN-Kα,, ol) 2: whle g > ol do 3: B mar blocks: B = 1 w )I and B j = w ji 4: D mar block: D, = α 2 f ) + 21 w )I 5: Echange eraes wh neghbors j N. 6: Graden: g = 1 w ) j N w j j + α f ). 7: Compue descen drecon d ) 8: for k =,..., K 1 do 9: Echange elemens d k) [ 1: NN-k + 1) sep: d k+1) 11: end for 12: Updae local erae: = + ɛ d K). 13: end whle = D 1 g of he NN-k sep wh neghbors B jd k) j g ]. j N,j= node. The graden componen g, = 1 w ), j N w j j, + α f,) s also sored and compued a. Node can also evaluae he values of he mar blocks B j = w ji. Thus, f he NN-k sep componens d k) j, are avalable a neghborng nodes j, node can hen deermne he NN-k + 1) sep componen d k+1), upon beng communcaed ha nformaon. The epresson n 17) represens an erave compuaon embedded nsde he NN-K recurson n 15). For each me nde, we compue he local componen of he sep d ), = D 1,g,. Upon echangng hs nformaon wh neghbors we use 17) o deermne he sep componens d 1),. These can be echanged and plugged n 17) o compue d 2),. Repeang hs procedure K mes, nodes ends up havng deermned her NN-K sep componen d K),. The NN-K mehod s summarzed n Algorhm 1. The descen eraon n 15) s mplemened n Sep 9. Implemenaon of hs descen requres access o he NN-K descen drecon d K), whch s compued by he loop n seps 4-8. Sep 4 nalzes he loop by compung he sep d ), = D 1,g,. The core of he loop s n Sep 7 whch corresponds o he recurson n 17). Sep 6 sands for he varable echange ha s necessary o mplemen Sep 7. Afer K eraons hrough hs loop he NN-K descen drecon d K), s compued and can be used n Sep 9. Boh, seps 4 and 9, requre access o he local graden componen g,. Ths s evaluaed n Sep 3 afer recevng he prerequse nformaon n Sep 2. B. Adapve Nework Newon As menoned n Secon II, NN-K algorhm nsead of solvng 1) or s equvalen 7), solves a penaly verson of 7) as nroduced n 4). The opmal soluons of opmzaon problems 7) and 4) are dfferen and he gap beween hem s upper bounded by Oα) [8]. Ths observaon mples ha by seng a decreasng polcy for α or equvalenly an ncreasng polcy for penaly coeffcen 1/α, he soluon of 7) approaches he mnmzer of 4),.e. ỹ y for α. We nroduce Adapve Nework Newon-K ANN-K) as a verson of NN-K ha uses a decreasng sequence of α o acheve eac convergence o he opmal soluon of 1). The dea of ANN-K s o decrease parameer α by mulplyng by η < 1,.e., α +1 = ηα, when he sequence generaed by NN-K s converged for a specfc value of α. To be more precse, each node has a sgnal vecor s = [s 1;... ; s n] {, 1} n where each componen s a bnary varable. Noe ha s j corresponds o he occurrence of recevng a sgnal a node from node j. Hence, nodes nalze her sgnalng componens by for all he nodes n he nework. A eraon node compues s local graden norm g,. If he norm of graden s smaller han a specfc value called ol,.e. g, ol, ses he local sgnal componen o s = 1 and sends a sgnal o all he nodes n he nework. The recever Algorhm 3 Adapve Nework Newon-K mehod a node Requre: Inal erae,, nal penaly parameer α and nal sequence of bs s = [s 1;... ; s n] = [;... ; ]. 1: for =, 1, 2,... do 2: Call NN-K funcon:,+1 = NN-Kα,,, ol) 3: Se s = 1 and broadcas scalar o all nodes. 4: Se s j = 1 for all nodes j ha sen a sgnal. 5: f s j = 1 for all j = 1,..., n hen 6: Updae penaly parameer α +1 = ηα. 7: Se s j = for all j = 1,..., n. 8: end f 9: end for nodes se he correspondng componen of node n her local sgnal vecors o 1,.e. s j = 1 for j. Ths procedure mples ha he sgnal vecors of all nodes n he nework are always synchronous. The updae for parameer α occurs when all he componens of sgnal vecor are 1 whch s equvalen o achevng he requred accuracy for all nodes n he nework. Snce he number of mes ha α should be updaed s small, he cos of communcaon for updang α s affordable. The ANN-K mehod s summarzed n Algorhm 3. A each eraon of ANN-K algorhm a Sep 2 funcon NN-K Sep s called o updae varable, for node. Noe ha funcon NN-K whch s nroduced n Algorhm 2, runs NN-K sep unl he me ha norm of local graden s smaller han a hreshold g ol. Afer achevng hs accuracy, n Seps 3 node updaes s local sgnal componen s o 1 and sends o he oher nodes. In Sep 4 each node updaes he sgnal vecor componens of oher nodes n he nework. Then, n Sep 6 he nodes updae he penaly parameer for he ne eraon as α +1 = ηα f all he componens of sgnal vecor s 1, oherwse hey use he prevous value α +1 = α. In order o rese he sysem afer updang α, all sgnal vecors are se o,.e. s = for = 1,..., n as n Sep 7. III. CONVERGENCE ANALYSIS In hs secon we show ha as me progresses he sequence of objecve funcon F y ) defned n 4) approaches he opmal objecve funcon value F y ) by consderng he followng assumpons. Assumpon 1 There ess consans δ < < 1 ha lower and upper bound he dagonal weghs for all, δ w < 1 = 1,..., n. 18) Assumpon 2 The egenvalues of local objecve funcon Hessans 2 f ) are bounded wh posve consans < m M <,.e. mi 2 f ) MI. 19) Assumpon 3 The local objecve funcon Hessans 2 f ) are Lpschz connuous wh parameer L wh respec o Eucldan norm, 2 f ) 2 f ˆ) L ˆ. 2) Lnear convergence of objecve funcon F y ) o he opmal objecve funcon F y ) s shown n [14] whch we menon as a reference. Theorem 1 Consder he NN-K mehod as defned n 1)-15) and he objecve funcon F y) as nroduced n 4). If he sepsze ɛ s chosen as ɛ = mn {1, ɛ } where ɛ s a consan ha depends on problem parameers, and Assumpons 1, 2, and 3 hold rue, he sequence F y ) converges o he opmal argumen F y ) a leas lnearly wh consan < 1 ζ < 1. I.e., F y ) F y ) 1 ζ) F y ) F y )). 21) Theorem 1 shows lnear convergence of sequence of objecve funcon F y ). In he followng secon we sudy he performances of NN and ANN mehods va dfferen numercal epermens.

4 Number of eraons Fg. 1: Convergence of,,, and n erms of number of eraons. The NN mehods converges faser han. Furhermore, he larger K s, he faser NN-K converges Number of local nformaon echanges Fg. 2: Convergence of,,, and n erms of number of communcaon echanges. The NN-K mehods rean he advanage over bu ncreasng K may no resul n faser convergence. For hs parcular nsance s acually ha converges fases n erms of number of communcaon echanges. IV. NUMERICAL ANALYSIS We compare he performance of and dfferen versons of NN n he mnmzaon of a dsrbued quadrac objecve. The comparson s done n erms of boh, number of eraons and number of nformaon echanges. Specfcally, for each agen we consder a posve defne dagonal mar A S ++ p and a vecor b R p o defne he local objecve funcon f ) := 1/2) T A + b T. Therefore, he global cos funcon f) s wren as f) := 1 2 T A + b T. 22) The dffculy of solvng 22) s gven by he condon number of he marces A. To adjus condon numbers we generae dagonal marces A wh random dagonal elemens a. The frs p/2 dagonal elemens a are drawn unformly a random from he dscree se {1, 1 1,..., 1 ξ } and he ne p/2 are unformly and randomly chosen from he se {1, 1 1,..., 1 ξ }. Ths choce of coeffcens yelds local marces A wh egenvalues n he nerval [1 ξ, 1 ξ ] and global marces A wh egenvalues n he nerval [n1 ξ, n1 ξ ]. The condon numbers are ypcally 1 2ξ for he local funcons and 1 ξ for he global objecves. The lnear erms b T are added so ha he dfferen local funcons have dfferen mnma. The vecors b are chosen unformly a random from he bo [, 1] p. For he quadrac objecve n 22) we can compue he opmal argumen n closed form. We hen evaluae convergence hrough he relave error ha we defne as he average normalzed squared dsance beween local vecors and he opmal decson vecor, e := 1, 2. 23) n The nework connecng he nodes s a d-regular cycle where each node s conneced o eacly d neghbors and d s assumed even. The graph s generaed by creang a cycle and hen connecng each node wh he d/2 nodes ha are closes n each drecon. The dagonal weghs n he mar W are se o w = 1/2 + 1/2d + 1) and he off dagonal weghs o w j = 1/2d + 1) when j N. In he subsequen epermens we se he nework sze o n = 1, he dmenson of he decson vecors o p = 4, he condon number parameer o ξ = 2, he penaly coeffcen nverse o α = 1 2, and he nework degree o d = 4. The NN sep sze s se o ɛ = 1, whch s always possble when we have quadrac objecves. Fgure 1 llusraes a sample convergence pah for,,, and by measurng he relave error e n 23) wh respec o he number of eraons. As epeced for a problem ha doesn have a small condon number n Emprcal dsrbuon Emprcal dsrbuon Number of nformaon echanges Number of nformaon echanges Emprcal dsrbuon Emprcal dsrbuon Number of nformaon echanges Number of nformaon echanges Fg. 3: Hsograms of he number of nformaon echanges requred o achevng accuracy e < 1 2. The qualave observaons made n fgures 1 and 2 hold over a range of random problem realzaons. hs parcular nsanaon of he funcon n 22) he condon number s 95.2 dfferen versons of NN are much faser han. E.g., afer = eraons he error assocaed whch eraes s e Comparable or beer accuracy e < s acheved n = 132, = 63, and = 43 eraons for,, and, respecvely. Furher recall ha α conrols he dfference beween he acual opmal argumen ỹ = [ ;... ; ] [cf. 7)] and he argumen y [cf. 4)] o whch and NN converge. Snce we have α = 1 2 and he dfference beween hese wo vecors s of order Oα), we epec he error n 23) o sele a e 1 2. The error acually seles a e and akes all hree versons of NN less han = 4 eraons o do so. I akes more han = 1 4 eraons o reach hs value. Ths relave performance dfference decreases f he problem has beer condonng bu can be made arbrarly large by ncreasng he condon number of he mar A. The number of eraons requred for convergence can be furher decreased by consderng hgher order appromaons n 14). The advanages would be msleadng because hey come a he cos of ncreasng he number of communcaons requred o appromae he Newon sep. To sudy hs laer effec we consder he relave performance of and dfferen versons of NN n erms of he number of local nformaon echanges. Noe ha each eraon n NN-K requres a oal of K + 1 nformaon echanges wh each neghbor, as opposed o he sngle

5 Number of eraons Fg. 4: Convergence of adapve,,, and for α = Number of eraons Fg. 5:Convergence of Adapve,,, and for α =1 1. varable echange requred by. Afer eraons he number of varable echanges beween each par of neghbors s for and K +1) for NN-K. Thus, we can ranslae Fgure 1 no a pah n erms of number of communcaons by scalng he me as by K + 1). The resul of hs scalng s shown n Fgure 2. The dfferen versons of NN rean a sgnfcan, albe smaller, advanage wh respec o. Error e < 1 2 s acheved by,, and afer K + 1) = , K + 1) = , and K + 1) = varable echanges, respecvely. When measured n hs merc s no longer rue ha ncreasng K resuls n faser convergence. For hs parcular problem nsance s acually ha converges fases n erms of number of communcaon echanges. For a more more comprehensve evaluaon we consder 1 3 dfferen random realzaons of 22) where we also randomze he degree d of he d-regular graph ha we choose from he even numbers n he se [2, 1]. The remanng parameers are he same used o generae fgures 1 and 2. For each jon random realzaon of nework and objecve we run,,, and, unl achevng error e < 1 2 and record he number of communcaon echanges ha have elapsed whch amoun o smply for and K + 1) for NN. The resulng hsograms are shown n Fgure 3. The mean mes requred o reduce he error o e < 1 2 are for and , , and for,, and. As n he parcular case shown n fgures 1 and 2, performs bes n erms of communcaon echanges. Observe, however, ha he number of communcaon echanges requred by s no much larger and ha requres less compuaonal effor han because he number of eraons s smaller. A. Adapve Nework Newon Gven ha and NN are penaly mehods s of neres o consder her behavor when he nverse penaly parameer α s decreased recursvely. The adapaon of α for NN-K s dscussed n Secon II-B where s ermed adapve A)NN-K. The same adapaon sraegy s consdered here for. The parameer α s kep consan unl he local graden componens g, become smaller han a gven olerance ol,.e., unl g, ol for all. When hs olerance s acheved, he parameer α s scaled by a facor η < 1,.e., α s decreased from s curren value o ηα. Ths requres he use of a sgnalng mehod lke he one summarzed n Algorhm 3 for ANN-K. We consder he objecve n 22) and nodes conneced by a d-regular cycle. We use he same parameers used o generae fgures 1 and 2. The adapve graden olerance s se o ol = 1 3 and he scalng parameer o η =.1. We consder wo dfferen scenaros where he nal penaly parameers are α = α = 1 1 and α = α = 1 2. The respecve error rajecores e wh respec o he number o eraons are shown n fgures 4 where α = 1 2 and 5 where α = 1 1. In each fgure we show e for adapve, A, A, and A. Boh fgures show ha he ANN mehods ouperform adapve and ha larger K reduces he number of eraons ha akes ANN-K o acheve a arge error. These resuls are conssen wh he fndngs summarzed n fgures 1-3. More neresng conclusons follow from a comparson across fgures 1 and 2. We can see ha s beer o sar wh he larger) value α = 1 1 even f he mehod nally converges o a pon farher from he acually opmum. Ths happens because problems wh larger α are beer condoned and hus easer o mnmze. REFERENCES [1] A. Rbero, Ergodc sochasc opmzaon algorhms for wreless communcaon and neworkng, IEEE Trans. Sgnal Process.., vol. 58, no. 12, pp , December 21. [2], Opmal resource allocaon n wreless communcaon and neworkng, EURASIP J. Wreless commun., vol. 212, no. 272, pp , Augus 212, pua carajo. [3] I. Schzas, A. Rbero, and G. Gannaks, Consensus n ad hoc wsns wh nosy lnks - par : Dsrbued esmaon of deermnsc sgnals, IEEE Transacons on Sgnal Processng, vol. 56, pp , 28. [4] M. Rabba and R. Nowak, Dsrbued opmzaon n sensor neworks, proceedngs of he 3rd nernaonal symposum on Informaon processng n sensor neworks, pp. 2 27, ACM, 24. [5] V. Cevher, S. Becker, and M. Schmd, Conve opmzaon for bg daa: Scalable, randomzed, and parallel algorhms for bg daa analycs, IEEE Sgnal Processng Magazne, vol. 31, pp , 214. [6] A. Nedc and A. Ozdaglar, Dsrbued subgraden mehods for mulagen opmzaon, IEEE Transacons on Auomac Conrol, vol. 54, pp , 29. [7] D. Jakovec, J. Xaver, and J. Moura, Fas dsrbued graden mehods, IEEE Transacons on Auomac Conrol, vol. 59, pp , 214. [8] K. Yuan, Q. Lng, and W. Yn, On he convergence of decenralzed graden descen, arxv preprn arxv, , 213. [9] W. Sh, Q. Lng, G. Wu, and W. Yn, Era: An eac frs-order algorhm for decenralzed consensus opmzaon, arxv preprn arxv, [1] Q. Lng and A. Rbero, Decenralzed lnearzed alernang drecon mehod of mulplers, Proc. In. Conf. Acouscs Speech Sgnal Process., pp , 214. [11] S. Boyd, N. Parkh, E. Chu, B. Peleao, and J. Ecksen, Dsrbued opmzaon and sascal learnng va he alernang drecon mehod of mulplers, Foundaons and Trends n Machne Learnng, vol. 3, no. 1, pp , 211. [12] W. Sh, Q. Lng, G. Wu, and W. Yn, On he lnear convergence of he admm n decenralzed consensus opmzaon, IEEE Transacons on Sgnal Processng, vol. 62, pp , 214. [13] J. Duch, A. Agarwal, and M. Wanwrgh, Dual averagng for dsrbued opmzaon: Convergence analyss and nework scalng, IEEE Transacons on Auomac Conrol, vol. 57, pp , 212. [14] A. Mokhar, Q. Lng, and A. Rbero, An appromae newon mehod for dsrbued opmzaon, 214, avalable a hp:// aryanm/wk/nn-icassp.pdf.

Variants of Pegasos. December 11, 2009

Variants of Pegasos. December 11, 2009 Inroducon Varans of Pegasos SooWoong Ryu bshboy@sanford.edu December, 009 Youngsoo Cho yc344@sanford.edu Developng a new SVM algorhm s ongong research opc. Among many exng SVM algorhms, we wll focus on

More information

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 4

CS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 4 CS434a/54a: Paern Recognon Prof. Olga Veksler Lecure 4 Oulne Normal Random Varable Properes Dscrmnan funcons Why Normal Random Varables? Analycally racable Works well when observaon comes form a corruped

More information

Solution in semi infinite diffusion couples (error function analysis)

Solution in semi infinite diffusion couples (error function analysis) Soluon n sem nfne dffuson couples (error funcon analyss) Le us consder now he sem nfne dffuson couple of wo blocks wh concenraon of and I means ha, n a A- bnary sysem, s bondng beween wo blocks made of

More information

John Geweke a and Gianni Amisano b a Departments of Economics and Statistics, University of Iowa, USA b European Central Bank, Frankfurt, Germany

John Geweke a and Gianni Amisano b a Departments of Economics and Statistics, University of Iowa, USA b European Central Bank, Frankfurt, Germany Herarchcal Markov Normal Mxure models wh Applcaons o Fnancal Asse Reurns Appendx: Proofs of Theorems and Condonal Poseror Dsrbuons John Geweke a and Gann Amsano b a Deparmens of Economcs and Sascs, Unversy

More information

Robustness Experiments with Two Variance Components

Robustness Experiments with Two Variance Components Naonal Insue of Sandards and Technology (NIST) Informaon Technology Laboraory (ITL) Sascal Engneerng Dvson (SED) Robusness Expermens wh Two Varance Componens by Ana Ivelsse Avlés avles@ns.gov Conference

More information

An introduction to Support Vector Machine

An introduction to Support Vector Machine An nroducon o Suppor Vecor Machne 報告者 : 黃立德 References: Smon Haykn, "Neural Neworks: a comprehensve foundaon, second edon, 999, Chaper 2,6 Nello Chrsann, John Shawe-Tayer, An Inroducon o Suppor Vecor Machnes,

More information

Volatility Interpolation

Volatility Interpolation Volaly Inerpolaon Prelmnary Verson March 00 Jesper Andreasen and Bran Huge Danse Mares, Copenhagen wan.daddy@danseban.com brno@danseban.com Elecronc copy avalable a: hp://ssrn.com/absrac=69497 Inro Local

More information

Lecture VI Regression

Lecture VI Regression Lecure VI Regresson (Lnear Mehods for Regresson) Conens: Lnear Mehods for Regresson Leas Squares, Gauss Markov heorem Recursve Leas Squares Lecure VI: MLSC - Dr. Sehu Vjayakumar Lnear Regresson Model M

More information

Dynamic Team Decision Theory. EECS 558 Project Shrutivandana Sharma and David Shuman December 10, 2005

Dynamic Team Decision Theory. EECS 558 Project Shrutivandana Sharma and David Shuman December 10, 2005 Dynamc Team Decson Theory EECS 558 Proec Shruvandana Sharma and Davd Shuman December 0, 005 Oulne Inroducon o Team Decson Theory Decomposon of he Dynamc Team Decson Problem Equvalence of Sac and Dynamc

More information

Cubic Bezier Homotopy Function for Solving Exponential Equations

Cubic Bezier Homotopy Function for Solving Exponential Equations Penerb Journal of Advanced Research n Compung and Applcaons ISSN (onlne: 46-97 Vol. 4, No.. Pages -8, 6 omoopy Funcon for Solvng Eponenal Equaons S. S. Raml *,,. Mohamad Nor,a, N. S. Saharzan,b and M.

More information

Machine Learning Linear Regression

Machine Learning Linear Regression Machne Learnng Lnear Regresson Lesson 3 Lnear Regresson Bascs of Regresson Leas Squares esmaon Polynomal Regresson Bass funcons Regresson model Regularzed Regresson Sascal Regresson Mamum Lkelhood (ML)

More information

V.Abramov - FURTHER ANALYSIS OF CONFIDENCE INTERVALS FOR LARGE CLIENT/SERVER COMPUTER NETWORKS

V.Abramov - FURTHER ANALYSIS OF CONFIDENCE INTERVALS FOR LARGE CLIENT/SERVER COMPUTER NETWORKS R&RATA # Vol.) 8, March FURTHER AALYSIS OF COFIDECE ITERVALS FOR LARGE CLIET/SERVER COMPUTER ETWORKS Vyacheslav Abramov School of Mahemacal Scences, Monash Unversy, Buldng 8, Level 4, Clayon Campus, Wellngon

More information

Advanced Machine Learning & Perception

Advanced Machine Learning & Perception Advanced Machne Learnng & Percepon Insrucor: Tony Jebara SVM Feaure & Kernel Selecon SVM Eensons Feaure Selecon (Flerng and Wrappng) SVM Feaure Selecon SVM Kernel Selecon SVM Eensons Classfcaon Feaure/Kernel

More information

In the complete model, these slopes are ANALYSIS OF VARIANCE FOR THE COMPLETE TWO-WAY MODEL. (! i+1 -! i ) + [(!") i+1,q - [(!

In the complete model, these slopes are ANALYSIS OF VARIANCE FOR THE COMPLETE TWO-WAY MODEL. (! i+1 -! i ) + [(!) i+1,q - [(! ANALYSIS OF VARIANCE FOR THE COMPLETE TWO-WAY MODEL The frs hng o es n wo-way ANOVA: Is here neracon? "No neracon" means: The man effecs model would f. Ths n urn means: In he neracon plo (wh A on he horzonal

More information

Lecture 6: Learning for Control (Generalised Linear Regression)

Lecture 6: Learning for Control (Generalised Linear Regression) Lecure 6: Learnng for Conrol (Generalsed Lnear Regresson) Conens: Lnear Mehods for Regresson Leas Squares, Gauss Markov heorem Recursve Leas Squares Lecure 6: RLSC - Prof. Sehu Vjayakumar Lnear Regresson

More information

CHAPTER 10: LINEAR DISCRIMINATION

CHAPTER 10: LINEAR DISCRIMINATION CHAPER : LINEAR DISCRIMINAION Dscrmnan-based Classfcaon 3 In classfcaon h K classes (C,C,, C k ) We defned dscrmnan funcon g j (), j=,,,k hen gven an es eample, e chose (predced) s class label as C f g

More information

Notes on the stability of dynamic systems and the use of Eigen Values.

Notes on the stability of dynamic systems and the use of Eigen Values. Noes on he sabl of dnamc ssems and he use of Egen Values. Source: Macro II course noes, Dr. Davd Bessler s Tme Seres course noes, zarads (999) Ineremporal Macroeconomcs chaper 4 & Techncal ppend, and Hamlon

More information

GENERATING CERTAIN QUINTIC IRREDUCIBLE POLYNOMIALS OVER FINITE FIELDS. Youngwoo Ahn and Kitae Kim

GENERATING CERTAIN QUINTIC IRREDUCIBLE POLYNOMIALS OVER FINITE FIELDS. Youngwoo Ahn and Kitae Kim Korean J. Mah. 19 (2011), No. 3, pp. 263 272 GENERATING CERTAIN QUINTIC IRREDUCIBLE POLYNOMIALS OVER FINITE FIELDS Youngwoo Ahn and Kae Km Absrac. In he paper [1], an explc correspondence beween ceran

More information

( t) Outline of program: BGC1: Survival and event history analysis Oslo, March-May Recapitulation. The additive regression model

( t) Outline of program: BGC1: Survival and event history analysis Oslo, March-May Recapitulation. The additive regression model BGC1: Survval and even hsory analyss Oslo, March-May 212 Monday May 7h and Tuesday May 8h The addve regresson model Ørnulf Borgan Deparmen of Mahemacs Unversy of Oslo Oulne of program: Recapulaon Counng

More information

Robust and Accurate Cancer Classification with Gene Expression Profiling

Robust and Accurate Cancer Classification with Gene Expression Profiling Robus and Accurae Cancer Classfcaon wh Gene Expresson Proflng (Compuaonal ysems Bology, 2005) Auhor: Hafeng L, Keshu Zhang, ao Jang Oulne Background LDA (lnear dscrmnan analyss) and small sample sze problem

More information

[ ] 2. [ ]3 + (Δx i + Δx i 1 ) / 2. Δx i-1 Δx i Δx i+1. TPG4160 Reservoir Simulation 2018 Lecture note 3. page 1 of 5

[ ] 2. [ ]3 + (Δx i + Δx i 1 ) / 2. Δx i-1 Δx i Δx i+1. TPG4160 Reservoir Simulation 2018 Lecture note 3. page 1 of 5 TPG460 Reservor Smulaon 08 page of 5 DISCRETIZATIO OF THE FOW EQUATIOS As we already have seen, fne dfference appromaons of he paral dervaves appearng n he flow equaons may be obaned from Taylor seres

More information

CS286.2 Lecture 14: Quantum de Finetti Theorems II

CS286.2 Lecture 14: Quantum de Finetti Theorems II CS286.2 Lecure 14: Quanum de Fne Theorems II Scrbe: Mara Okounkova 1 Saemen of he heorem Recall he las saemen of he quanum de Fne heorem from he prevous lecure. Theorem 1 Quanum de Fne). Le ρ Dens C 2

More information

CH.3. COMPATIBILITY EQUATIONS. Continuum Mechanics Course (MMC) - ETSECCPB - UPC

CH.3. COMPATIBILITY EQUATIONS. Continuum Mechanics Course (MMC) - ETSECCPB - UPC CH.3. COMPATIBILITY EQUATIONS Connuum Mechancs Course (MMC) - ETSECCPB - UPC Overvew Compably Condons Compably Equaons of a Poenal Vecor Feld Compably Condons for Infnesmal Srans Inegraon of he Infnesmal

More information

Lecture 18: The Laplace Transform (See Sections and 14.7 in Boas)

Lecture 18: The Laplace Transform (See Sections and 14.7 in Boas) Lecure 8: The Lalace Transform (See Secons 88- and 47 n Boas) Recall ha our bg-cure goal s he analyss of he dfferenal equaon, ax bx cx F, where we emloy varous exansons for he drvng funcon F deendng on

More information

Ordinary Differential Equations in Neuroscience with Matlab examples. Aim 1- Gain understanding of how to set up and solve ODE s

Ordinary Differential Equations in Neuroscience with Matlab examples. Aim 1- Gain understanding of how to set up and solve ODE s Ordnary Dfferenal Equaons n Neuroscence wh Malab eamples. Am - Gan undersandng of how o se up and solve ODE s Am Undersand how o se up an solve a smple eample of he Hebb rule n D Our goal a end of class

More information

Department of Economics University of Toronto

Department of Economics University of Toronto Deparmen of Economcs Unversy of Torono ECO408F M.A. Economercs Lecure Noes on Heeroskedascy Heeroskedascy o Ths lecure nvolves lookng a modfcaons we need o make o deal wh he regresson model when some of

More information

Clustering (Bishop ch 9)

Clustering (Bishop ch 9) Cluserng (Bshop ch 9) Reference: Daa Mnng by Margare Dunham (a slde source) 1 Cluserng Cluserng s unsupervsed learnng, here are no class labels Wan o fnd groups of smlar nsances Ofen use a dsance measure

More information

TSS = SST + SSE An orthogonal partition of the total SS

TSS = SST + SSE An orthogonal partition of the total SS ANOVA: Topc 4. Orhogonal conrass [ST&D p. 183] H 0 : µ 1 = µ =... = µ H 1 : The mean of a leas one reamen group s dfferen To es hs hypohess, a basc ANOVA allocaes he varaon among reamen means (SST) equally

More information

THE PREDICTION OF COMPETITIVE ENVIRONMENT IN BUSINESS

THE PREDICTION OF COMPETITIVE ENVIRONMENT IN BUSINESS THE PREICTION OF COMPETITIVE ENVIRONMENT IN BUSINESS INTROUCTION The wo dmensonal paral dfferenal equaons of second order can be used for he smulaon of compeve envronmen n busness The arcle presens he

More information

FTCS Solution to the Heat Equation

FTCS Solution to the Heat Equation FTCS Soluon o he Hea Equaon ME 448/548 Noes Gerald Reckenwald Porland Sae Unversy Deparmen of Mechancal Engneerng gerry@pdxedu ME 448/548: FTCS Soluon o he Hea Equaon Overvew Use he forward fne d erence

More information

On One Analytic Method of. Constructing Program Controls

On One Analytic Method of. Constructing Program Controls Appled Mahemacal Scences, Vol. 9, 05, no. 8, 409-407 HIKARI Ld, www.m-hkar.com hp://dx.do.org/0.988/ams.05.54349 On One Analyc Mehod of Consrucng Program Conrols A. N. Kvko, S. V. Chsyakov and Yu. E. Balyna

More information

J i-1 i. J i i+1. Numerical integration of the diffusion equation (I) Finite difference method. Spatial Discretization. Internal nodes.

J i-1 i. J i i+1. Numerical integration of the diffusion equation (I) Finite difference method. Spatial Discretization. Internal nodes. umercal negraon of he dffuson equaon (I) Fne dfference mehod. Spaal screaon. Inernal nodes. R L V For hermal conducon le s dscree he spaal doman no small fne spans, =,,: Balance of parcles for an nernal

More information

Relative controllability of nonlinear systems with delays in control

Relative controllability of nonlinear systems with delays in control Relave conrollably o nonlnear sysems wh delays n conrol Jerzy Klamka Insue o Conrol Engneerng, Slesan Techncal Unversy, 44- Glwce, Poland. phone/ax : 48 32 37227, {jklamka}@a.polsl.glwce.pl Keywor: Conrollably.

More information

Reactive Methods to Solve the Berth AllocationProblem with Stochastic Arrival and Handling Times

Reactive Methods to Solve the Berth AllocationProblem with Stochastic Arrival and Handling Times Reacve Mehods o Solve he Berh AllocaonProblem wh Sochasc Arrval and Handlng Tmes Nsh Umang* Mchel Berlare* * TRANSP-OR, Ecole Polyechnque Fédérale de Lausanne Frs Workshop on Large Scale Opmzaon November

More information

( ) () we define the interaction representation by the unitary transformation () = ()

( ) () we define the interaction representation by the unitary transformation () = () Hgher Order Perurbaon Theory Mchael Fowler 3/7/6 The neracon Represenaon Recall ha n he frs par of hs course sequence, we dscussed he chrödnger and Hesenberg represenaons of quanum mechancs here n he chrödnger

More information

. The geometric multiplicity is dim[ker( λi. number of linearly independent eigenvectors associated with this eigenvalue.

. The geometric multiplicity is dim[ker( λi. number of linearly independent eigenvectors associated with this eigenvalue. Lnear Algebra Lecure # Noes We connue wh he dscusson of egenvalues, egenvecors, and dagonalzably of marces We wan o know, n parcular wha condons wll assure ha a marx can be dagonalzed and wha he obsrucons

More information

MANY real-world applications (e.g. production

MANY real-world applications (e.g. production Barebones Parcle Swarm for Ineger Programmng Problems Mahamed G. H. Omran, Andres Engelbrech and Ayed Salman Absrac The performance of wo recen varans of Parcle Swarm Opmzaon (PSO) when appled o Ineger

More information

EEL 6266 Power System Operation and Control. Chapter 5 Unit Commitment

EEL 6266 Power System Operation and Control. Chapter 5 Unit Commitment EEL 6266 Power Sysem Operaon and Conrol Chaper 5 Un Commmen Dynamc programmng chef advanage over enumeraon schemes s he reducon n he dmensonaly of he problem n a src prory order scheme, here are only N

More information

F-Tests and Analysis of Variance (ANOVA) in the Simple Linear Regression Model. 1. Introduction

F-Tests and Analysis of Variance (ANOVA) in the Simple Linear Regression Model. 1. Introduction ECOOMICS 35* -- OTE 9 ECO 35* -- OTE 9 F-Tess and Analyss of Varance (AOVA n he Smple Lnear Regresson Model Inroducon The smple lnear regresson model s gven by he followng populaon regresson equaon, or

More information

Computing Relevance, Similarity: The Vector Space Model

Computing Relevance, Similarity: The Vector Space Model Compung Relevance, Smlary: The Vecor Space Model Based on Larson and Hears s sldes a UC-Bereley hp://.sms.bereley.edu/courses/s0/f00/ aabase Managemen Sysems, R. Ramarshnan ocumen Vecors v ocumens are

More information

Graduate Macroeconomics 2 Problem set 5. - Solutions

Graduate Macroeconomics 2 Problem set 5. - Solutions Graduae Macroeconomcs 2 Problem se. - Soluons Queson 1 To answer hs queson we need he frms frs order condons and he equaon ha deermnes he number of frms n equlbrum. The frms frs order condons are: F K

More information

Appendix H: Rarefaction and extrapolation of Hill numbers for incidence data

Appendix H: Rarefaction and extrapolation of Hill numbers for incidence data Anne Chao Ncholas J Goell C seh lzabeh L ander K Ma Rober K Colwell and Aaron M llson 03 Rarefacon and erapolaon wh ll numbers: a framewor for samplng and esmaon n speces dversy sudes cology Monographs

More information

Chapter Lagrangian Interpolation

Chapter Lagrangian Interpolation Chaper 5.4 agrangan Inerpolaon Afer readng hs chaper you should be able o:. dere agrangan mehod of nerpolaon. sole problems usng agrangan mehod of nerpolaon and. use agrangan nerpolans o fnd deraes and

More information

. The geometric multiplicity is dim[ker( λi. A )], i.e. the number of linearly independent eigenvectors associated with this eigenvalue.

. The geometric multiplicity is dim[ker( λi. A )], i.e. the number of linearly independent eigenvectors associated with this eigenvalue. Mah E-b Lecure #0 Noes We connue wh he dscusson of egenvalues, egenvecors, and dagonalzably of marces We wan o know, n parcular wha condons wll assure ha a marx can be dagonalzed and wha he obsrucons are

More information

DEEP UNFOLDING FOR MULTICHANNEL SOURCE SEPARATION SUPPLEMENTARY MATERIAL

DEEP UNFOLDING FOR MULTICHANNEL SOURCE SEPARATION SUPPLEMENTARY MATERIAL DEEP UNFOLDING FOR MULTICHANNEL SOURCE SEPARATION SUPPLEMENTARY MATERIAL Sco Wsdom, John Hershey 2, Jonahan Le Roux 2, and Shnj Waanabe 2 Deparmen o Elecrcal Engneerng, Unversy o Washngon, Seale, WA, USA

More information

Single-loop System Reliability-Based Design & Topology Optimization (SRBDO/SRBTO): A Matrix-based System Reliability (MSR) Method

Single-loop System Reliability-Based Design & Topology Optimization (SRBDO/SRBTO): A Matrix-based System Reliability (MSR) Method 10 h US Naonal Congress on Compuaonal Mechancs Columbus, Oho 16-19, 2009 Sngle-loop Sysem Relably-Based Desgn & Topology Opmzaon (SRBDO/SRBTO): A Marx-based Sysem Relably (MSR) Mehod Tam Nguyen, Junho

More information

CHAPTER 5: MULTIVARIATE METHODS

CHAPTER 5: MULTIVARIATE METHODS CHAPER 5: MULIVARIAE MEHODS Mulvarae Daa 3 Mulple measuremens (sensors) npus/feaures/arbues: -varae N nsances/observaons/eamples Each row s an eample Each column represens a feaure X a b correspons o he

More information

New M-Estimator Objective Function. in Simultaneous Equations Model. (A Comparative Study)

New M-Estimator Objective Function. in Simultaneous Equations Model. (A Comparative Study) Inernaonal Mahemacal Forum, Vol. 8, 3, no., 7 - HIKARI Ld, www.m-hkar.com hp://dx.do.org/.988/mf.3.3488 New M-Esmaor Objecve Funcon n Smulaneous Equaons Model (A Comparave Sudy) Ahmed H. Youssef Professor

More information

Lecture 11 SVM cont

Lecture 11 SVM cont Lecure SVM con. 0 008 Wha we have done so far We have esalshed ha we wan o fnd a lnear decson oundary whose margn s he larges We know how o measure he margn of a lnear decson oundary Tha s: he mnmum geomerc

More information

arxiv: v1 [cs.sy] 2 Sep 2014

arxiv: v1 [cs.sy] 2 Sep 2014 Noname manuscrp No. wll be nsered by he edor Sgnalng for Decenralzed Roung n a Queueng Nework Y Ouyang Demoshens Tenekezs Receved: dae / Acceped: dae arxv:409.0887v [cs.sy] Sep 04 Absrac A dscree-me decenralzed

More information

Performance Analysis for a Network having Standby Redundant Unit with Waiting in Repair

Performance Analysis for a Network having Standby Redundant Unit with Waiting in Repair TECHNI Inernaonal Journal of Compung Scence Communcaon Technologes VOL.5 NO. July 22 (ISSN 974-3375 erformance nalyss for a Nework havng Sby edundan Un wh ang n epar Jendra Sngh 2 abns orwal 2 Deparmen

More information

Fall 2010 Graduate Course on Dynamic Learning

Fall 2010 Graduate Course on Dynamic Learning Fall 200 Graduae Course on Dynamc Learnng Chaper 4: Parcle Flers Sepember 27, 200 Byoung-Tak Zhang School of Compuer Scence and Engneerng & Cognve Scence and Bran Scence Programs Seoul aonal Unversy hp://b.snu.ac.kr/~bzhang/

More information

Math 128b Project. Jude Yuen

Math 128b Project. Jude Yuen Mah 8b Proec Jude Yuen . Inroducon Le { Z } be a sequence of observed ndependen vecor varables. If he elemens of Z have a on normal dsrbuon hen { Z } has a mean vecor Z and a varancecovarance marx z. Geomercally

More information

UNIVERSITAT AUTÒNOMA DE BARCELONA MARCH 2017 EXAMINATION

UNIVERSITAT AUTÒNOMA DE BARCELONA MARCH 2017 EXAMINATION INTERNATIONAL TRADE T. J. KEHOE UNIVERSITAT AUTÒNOMA DE BARCELONA MARCH 27 EXAMINATION Please answer wo of he hree quesons. You can consul class noes, workng papers, and arcles whle you are workng on he

More information

Online Supplement for Dynamic Multi-Technology. Production-Inventory Problem with Emissions Trading

Online Supplement for Dynamic Multi-Technology. Production-Inventory Problem with Emissions Trading Onlne Supplemen for Dynamc Mul-Technology Producon-Invenory Problem wh Emssons Tradng by We Zhang Zhongsheng Hua Yu Xa and Baofeng Huo Proof of Lemma For any ( qr ) Θ s easy o verfy ha he lnear programmng

More information

Dual Approximate Dynamic Programming for Large Scale Hydro Valleys

Dual Approximate Dynamic Programming for Large Scale Hydro Valleys Dual Approxmae Dynamc Programmng for Large Scale Hydro Valleys Perre Carpener and Jean-Phlppe Chanceler 1 ENSTA ParsTech and ENPC ParsTech CMM Workshop, January 2016 1 Jon work wh J.-C. Alas, suppored

More information

Outline. Probabilistic Model Learning. Probabilistic Model Learning. Probabilistic Model for Time-series Data: Hidden Markov Model

Outline. Probabilistic Model Learning. Probabilistic Model Learning. Probabilistic Model for Time-series Data: Hidden Markov Model Probablsc Model for Tme-seres Daa: Hdden Markov Model Hrosh Mamsuka Bonformacs Cener Kyoo Unversy Oulne Three Problems for probablsc models n machne learnng. Compung lkelhood 2. Learnng 3. Parsng (predcon

More information

Comparison of Differences between Power Means 1

Comparison of Differences between Power Means 1 In. Journal of Mah. Analyss, Vol. 7, 203, no., 5-55 Comparson of Dfferences beween Power Means Chang-An Tan, Guanghua Sh and Fe Zuo College of Mahemacs and Informaon Scence Henan Normal Unversy, 453007,

More information

A Deterministic Algorithm for Summarizing Asynchronous Streams over a Sliding Window

A Deterministic Algorithm for Summarizing Asynchronous Streams over a Sliding Window A Deermnsc Algorhm for Summarzng Asynchronous Sreams over a Sldng ndow Cosas Busch Rensselaer Polyechnc Insue Srkana Trhapura Iowa Sae Unversy Oulne of Talk Inroducon Algorhm Analyss Tme C Daa sream: 3

More information

Sampling Procedure of the Sum of two Binary Markov Process Realizations

Sampling Procedure of the Sum of two Binary Markov Process Realizations Samplng Procedure of he Sum of wo Bnary Markov Process Realzaons YURY GORITSKIY Dep. of Mahemacal Modelng of Moscow Power Insue (Techncal Unversy), Moscow, RUSSIA, E-mal: gorsky@yandex.ru VLADIMIR KAZAKOV

More information

Machine Learning 2nd Edition

Machine Learning 2nd Edition INTRODUCTION TO Lecure Sldes for Machne Learnng nd Edon ETHEM ALPAYDIN, modfed by Leonardo Bobadlla and some pars from hp://www.cs.au.ac.l/~aparzn/machnelearnng/ The MIT Press, 00 alpaydn@boun.edu.r hp://www.cmpe.boun.edu.r/~ehem/mle

More information

Li An-Ping. Beijing , P.R.China

Li An-Ping. Beijing , P.R.China A New Type of Cpher: DICING_csb L An-Png Bejng 100085, P.R.Chna apl0001@sna.com Absrac: In hs paper, we wll propose a new ype of cpher named DICING_csb, whch s derved from our prevous sream cpher DICING.

More information

Including the ordinary differential of distance with time as velocity makes a system of ordinary differential equations.

Including the ordinary differential of distance with time as velocity makes a system of ordinary differential equations. Soluons o Ordnary Derenal Equaons An ordnary derenal equaon has only one ndependen varable. A sysem o ordnary derenal equaons consss o several derenal equaons each wh he same ndependen varable. An eample

More information

HEAT CONDUCTION PROBLEM IN A TWO-LAYERED HOLLOW CYLINDER BY USING THE GREEN S FUNCTION METHOD

HEAT CONDUCTION PROBLEM IN A TWO-LAYERED HOLLOW CYLINDER BY USING THE GREEN S FUNCTION METHOD Journal of Appled Mahemacs and Compuaonal Mechancs 3, (), 45-5 HEAT CONDUCTION PROBLEM IN A TWO-LAYERED HOLLOW CYLINDER BY USING THE GREEN S FUNCTION METHOD Sansław Kukla, Urszula Sedlecka Insue of Mahemacs,

More information

GMM parameter estimation. Xiaoye Lu CMPS290c Final Project

GMM parameter estimation. Xiaoye Lu CMPS290c Final Project GMM paraeer esaon Xaoye Lu M290c Fnal rojec GMM nroducon Gaussan ure Model obnaon of several gaussan coponens Noaon: For each Gaussan dsrbuon:, s he ean and covarance ar. A GMM h ures(coponens): p ( 2π

More information

Joint Channel Estimation and Resource Allocation for MIMO Systems Part I: Single-User Analysis

Joint Channel Estimation and Resource Allocation for MIMO Systems Part I: Single-User Analysis 624 IEEE RANSACIONS ON WIRELESS COUNICAIONS, VOL. 9, NO. 2, FEBRUARY 200 Jon Channel Esmaon and Resource Allocaon for IO Sysems Par I: Sngle-User Analyss Alkan Soysal, ember, IEEE, and Sennur Ulukus, ember,

More information

Privacy Preserving Randomized Gossip Algorithms. University of Edinburgh, UK KAUST, KSA Higher School of Economics, Russia.

Privacy Preserving Randomized Gossip Algorithms. University of Edinburgh, UK KAUST, KSA Higher School of Economics, Russia. Prvacy Preservng Randomzed Gossp Algorhms Flp Hanzely Jakub Konečný Ncolas Lozou Peer Rchárk Dmry Grshchenko Unversy of dnburgh, UK KAUST, KSA Hgher School of conomcs, Russa June, 07 arxv:706.07636v mah.oc

More information

Tight results for Next Fit and Worst Fit with resource augmentation

Tight results for Next Fit and Worst Fit with resource augmentation Tgh resuls for Nex F and Wors F wh resource augmenaon Joan Boyar Leah Epsen Asaf Levn Asrac I s well known ha he wo smple algorhms for he classc n packng prolem, NF and WF oh have an approxmaon rao of

More information

Boosted LMS-based Piecewise Linear Adaptive Filters

Boosted LMS-based Piecewise Linear Adaptive Filters 016 4h European Sgnal Processng Conference EUSIPCO) Boosed LMS-based Pecewse Lnear Adapve Flers Darush Kar and Iman Marvan Deparmen of Elecrcal and Elecroncs Engneerng Blken Unversy, Ankara, Turkey {kar,

More information

This document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore.

This document is downloaded from DR-NTU, Nanyang Technological University Library, Singapore. Ths documen s downloaded from DR-NTU, Nanyang Technologcal Unversy Lbrary, Sngapore. Tle A smplfed verb machng algorhm for word paron n vsual speech processng( Acceped verson ) Auhor(s) Foo, Say We; Yong,

More information

Comb Filters. Comb Filters

Comb Filters. Comb Filters The smple flers dscussed so far are characered eher by a sngle passband and/or a sngle sopband There are applcaons where flers wh mulple passbands and sopbands are requred Thecomb fler s an example of

More information

Dynamic Team Decision Theory

Dynamic Team Decision Theory Dynamc Team Decson Theory EECS 558 Proec Repor Shruvandana Sharma and Davd Shuman December, 005 I. Inroducon Whle he sochasc conrol problem feaures one decson maker acng over me, many complex conrolled

More information

Bayes rule for a classification problem INF Discriminant functions for the normal density. Euclidean distance. Mahalanobis distance

Bayes rule for a classification problem INF Discriminant functions for the normal density. Euclidean distance. Mahalanobis distance INF 43 3.. Repeon Anne Solberg (anne@f.uo.no Bayes rule for a classfcaon problem Suppose we have J, =,...J classes. s he class label for a pxel, and x s he observed feaure vecor. We can use Bayes rule

More information

10. A.C CIRCUITS. Theoretically current grows to maximum value after infinite time. But practically it grows to maximum after 5τ. Decay of current :

10. A.C CIRCUITS. Theoretically current grows to maximum value after infinite time. But practically it grows to maximum after 5τ. Decay of current : . A. IUITS Synopss : GOWTH OF UNT IN IUIT : d. When swch S s closed a =; = d. A me, curren = e 3. The consan / has dmensons of me and s called he nducve me consan ( τ ) of he crcu. 4. = τ; =.63, n one

More information

SOME NOISELESS CODING THEOREMS OF INACCURACY MEASURE OF ORDER α AND TYPE β

SOME NOISELESS CODING THEOREMS OF INACCURACY MEASURE OF ORDER α AND TYPE β SARAJEVO JOURNAL OF MATHEMATICS Vol.3 (15) (2007), 137 143 SOME NOISELESS CODING THEOREMS OF INACCURACY MEASURE OF ORDER α AND TYPE β M. A. K. BAIG AND RAYEES AHMAD DAR Absrac. In hs paper, we propose

More information

A Simulation Based Optimal Control System For Water Resources

A Simulation Based Optimal Control System For Water Resources Cy Unversy of New York (CUNY) CUNY Academc Works Inernaonal Conference on Hydronformacs 8--4 A Smulaon Based Opmal Conrol Sysem For Waer Resources Aser acasa Maro Morales-Hernández Plar Brufau Plar García-Navarro

More information

Existence and Uniqueness Results for Random Impulsive Integro-Differential Equation

Existence and Uniqueness Results for Random Impulsive Integro-Differential Equation Global Journal of Pure and Appled Mahemacs. ISSN 973-768 Volume 4, Number 6 (8), pp. 89-87 Research Inda Publcaons hp://www.rpublcaon.com Exsence and Unqueness Resuls for Random Impulsve Inegro-Dfferenal

More information

WiH Wei He

WiH Wei He Sysem Idenfcaon of onlnear Sae-Space Space Baery odels WH We He wehe@calce.umd.edu Advsor: Dr. Chaochao Chen Deparmen of echancal Engneerng Unversy of aryland, College Par 1 Unversy of aryland Bacground

More information

Linear Response Theory: The connection between QFT and experiments

Linear Response Theory: The connection between QFT and experiments Phys540.nb 39 3 Lnear Response Theory: The connecon beween QFT and expermens 3.1. Basc conceps and deas Q: ow do we measure he conducvy of a meal? A: we frs nroduce a weak elecrc feld E, and hen measure

More information

2/20/2013. EE 101 Midterm 2 Review

2/20/2013. EE 101 Midterm 2 Review //3 EE Mderm eew //3 Volage-mplfer Model The npu ressance s he equalen ressance see when lookng no he npu ermnals of he amplfer. o s he oupu ressance. I causes he oupu olage o decrease as he load ressance

More information

Survival Analysis and Reliability. A Note on the Mean Residual Life Function of a Parallel System

Survival Analysis and Reliability. A Note on the Mean Residual Life Function of a Parallel System Communcaons n Sascs Theory and Mehods, 34: 475 484, 2005 Copyrgh Taylor & Francs, Inc. ISSN: 0361-0926 prn/1532-415x onlne DOI: 10.1081/STA-200047430 Survval Analyss and Relably A Noe on he Mean Resdual

More information

Chapter 4. Neural Networks Based on Competition

Chapter 4. Neural Networks Based on Competition Chaper 4. Neural Neworks Based on Compeon Compeon s mporan for NN Compeon beween neurons has been observed n bologcal nerve sysems Compeon s mporan n solvng many problems To classfy an npu paern _1 no

More information

( ) [ ] MAP Decision Rule

( ) [ ] MAP Decision Rule Announcemens Bayes Decson Theory wh Normal Dsrbuons HW0 due oday HW o be assgned soon Proec descrpon posed Bomercs CSE 90 Lecure 4 CSE90, Sprng 04 CSE90, Sprng 04 Key Probables 4 ω class label X feaure

More information

On computing differential transform of nonlinear non-autonomous functions and its applications

On computing differential transform of nonlinear non-autonomous functions and its applications On compung dfferenal ransform of nonlnear non-auonomous funcons and s applcaons Essam. R. El-Zahar, and Abdelhalm Ebad Deparmen of Mahemacs, Faculy of Scences and Humanes, Prnce Saam Bn Abdulazz Unversy,

More information

How about the more general "linear" scalar functions of scalars (i.e., a 1st degree polynomial of the following form with a constant term )?

How about the more general linear scalar functions of scalars (i.e., a 1st degree polynomial of the following form with a constant term )? lmcd Lnear ransformaon of a vecor he deas presened here are que general hey go beyond he radonal mar-vecor ype seen n lnear algebra Furhermore, hey do no deal wh bass and are equally vald for any se of

More information

Technical report a

Technical report a Delf Unversy of Technology Delf Cener for Sysems and Conrol Techncal repor 11-039a A dsrbued opmzaon-based approach for herarchcal model predcve conrol of large-scale sysems wh coupled dynamcs and consrans:

More information

COMPUTER SCIENCE 349A SAMPLE EXAM QUESTIONS WITH SOLUTIONS PARTS 1, 2

COMPUTER SCIENCE 349A SAMPLE EXAM QUESTIONS WITH SOLUTIONS PARTS 1, 2 COMPUTE SCIENCE 49A SAMPLE EXAM QUESTIONS WITH SOLUTIONS PATS, PAT.. a Dene he erm ll-ondoned problem. b Gve an eample o a polynomal ha has ll-ondoned zeros.. Consder evaluaon o anh, where e e anh. e e

More information

Advanced Macroeconomics II: Exchange economy

Advanced Macroeconomics II: Exchange economy Advanced Macroeconomcs II: Exchange economy Krzyszof Makarsk 1 Smple deermnsc dynamc model. 1.1 Inroducon Inroducon Smple deermnsc dynamc model. Defnons of equlbrum: Arrow-Debreu Sequenal Recursve Equvalence

More information

Optimal environmental charges under imperfect compliance

Optimal environmental charges under imperfect compliance ISSN 1 746-7233, England, UK World Journal of Modellng and Smulaon Vol. 4 (28) No. 2, pp. 131-139 Opmal envronmenal charges under mperfec complance Dajn Lu 1, Ya Wang 2 Tazhou Insue of Scence and Technology,

More information

Introduction to Boosting

Introduction to Boosting Inroducon o Boosng Cynha Rudn PACM, Prnceon Unversy Advsors Ingrd Daubeches and Rober Schapre Say you have a daabase of news arcles, +, +, -, -, +, +, -, -, +, +, -, -, +, +, -, + where arcles are labeled

More information

Approximate Analytic Solution of (2+1) - Dimensional Zakharov-Kuznetsov(Zk) Equations Using Homotopy

Approximate Analytic Solution of (2+1) - Dimensional Zakharov-Kuznetsov(Zk) Equations Using Homotopy Arcle Inernaonal Journal of Modern Mahemacal Scences, 4, (): - Inernaonal Journal of Modern Mahemacal Scences Journal homepage: www.modernscenfcpress.com/journals/jmms.aspx ISSN: 66-86X Florda, USA Approxmae

More information

[Link to MIT-Lab 6P.1 goes here.] After completing the lab, fill in the following blanks: Numerical. Simulation s Calculations

[Link to MIT-Lab 6P.1 goes here.] After completing the lab, fill in the following blanks: Numerical. Simulation s Calculations Chaper 6: Ordnary Leas Squares Esmaon Procedure he Properes Chaper 6 Oulne Cln s Assgnmen: Assess he Effec of Sudyng on Quz Scores Revew o Regresson Model o Ordnary Leas Squares () Esmaon Procedure o he

More information

Bandlimited channel. Intersymbol interference (ISI) This non-ideal communication channel is also called dispersive channel

Bandlimited channel. Intersymbol interference (ISI) This non-ideal communication channel is also called dispersive channel Inersymol nererence ISI ISI s a sgnal-dependen orm o nererence ha arses ecause o devaons n he requency response o a channel rom he deal channel. Example: Bandlmed channel Tme Doman Bandlmed channel Frequency

More information

A GENERAL FRAMEWORK FOR CONTINUOUS TIME POWER CONTROL IN TIME VARYING LONG TERM FADING WIRELESS NETWORKS

A GENERAL FRAMEWORK FOR CONTINUOUS TIME POWER CONTROL IN TIME VARYING LONG TERM FADING WIRELESS NETWORKS A GENERAL FRAEWORK FOR CONTINUOUS TIE POWER CONTROL IN TIE VARYING LONG TER FADING WIRELESS NETWORKS ohammed. Olama, Seddk. Djouad Charalambos D. Charalambous Elecrcal and Compuer Engneerng Deparmen Elecrcal

More information

GAME theory is a field of mathematics that studies conflict. Dynamic Potential Games with Constraints: Fundamentals and Applications in Communications

GAME theory is a field of mathematics that studies conflict. Dynamic Potential Games with Constraints: Fundamentals and Applications in Communications 1 Dynamc Poenal Games wh Consrans: Fundamenals and Applcaons n Communcaons Sanago Zazo, Member, IEEE, Sergo Valcarcel Macua, Suden Member, IEEE, Malde Sánchez-Fernández, Senor Member, IEEE, Javer Zazo

More information

e-journal Reliability: Theory& Applications No 2 (Vol.2) Vyacheslav Abramov

e-journal Reliability: Theory& Applications No 2 (Vol.2) Vyacheslav Abramov June 7 e-ournal Relably: Theory& Applcaons No (Vol. CONFIDENCE INTERVALS ASSOCIATED WITH PERFORMANCE ANALYSIS OF SYMMETRIC LARGE CLOSED CLIENT/SERVER COMPUTER NETWORKS Absrac Vyacheslav Abramov School

More information

CS 268: Packet Scheduling

CS 268: Packet Scheduling Pace Schedulng Decde when and wha pace o send on oupu ln - Usually mplemened a oupu nerface CS 68: Pace Schedulng flow Ion Soca March 9, 004 Classfer flow flow n Buffer managemen Scheduler soca@cs.bereley.edu

More information

2 Aggregate demand in partial equilibrium static framework

2 Aggregate demand in partial equilibrium static framework Unversy of Mnnesoa 8107 Macroeconomc Theory, Sprng 2009, Mn 1 Fabrzo Perr Lecure 1. Aggregaon 1 Inroducon Probably so far n he macro sequence you have deal drecly wh represenave consumers and represenave

More information

APOC #232 Capacity Planning for Fault-Tolerant All-Optical Network

APOC #232 Capacity Planning for Fault-Tolerant All-Optical Network APOC #232 Capacy Plannng for Faul-Toleran All-Opcal Nework Mchael Kwok-Shng Ho and Kwok-wa Cheung Deparmen of Informaon ngneerng The Chnese Unversy of Hong Kong Shan, N.T., Hong Kong SAR, Chna -mal: kwcheung@e.cuhk.edu.hk

More information

CS 536: Machine Learning. Nonparametric Density Estimation Unsupervised Learning - Clustering

CS 536: Machine Learning. Nonparametric Density Estimation Unsupervised Learning - Clustering CS 536: Machne Learnng Nonparamerc Densy Esmaon Unsupervsed Learnng - Cluserng Fall 2005 Ahmed Elgammal Dep of Compuer Scence Rugers Unversy CS 536 Densy Esmaon - Cluserng - 1 Oulnes Densy esmaon Nonparamerc

More information