A Derivative-Free Algorithm for Bound Constrained Optimization

Size: px
Start display at page:

Download "A Derivative-Free Algorithm for Bound Constrained Optimization"

Transcription

1 Computatona Optmzaton and Appcatons, 21, , 2002 c 2002 Kuwer Academc Pubshers. Manufactured n The Netherands. A Dervatve-Free Agorthm for Bound Constraned Optmzaton STEFANO LUCIDI ucd@ds.unroma.t Dpartmento d Informatca e Sstemstca, Unverstà d Roma La Sapenza, Roma, Itay MARCO SCIANDRONE Isttuto d Anas de Sstem ed Informatca, CNR, Roma, Itay scandro@as.rm.cnr.t Revsed 15 January, 2001 Abstract. In ths wor, we propose a new gobay convergent dervatve-free agorthm for the mnmzaton of a contnuousy dfferentabe functon n the case that some of (or a) the varabes are bounded. Ths agorthm nvestgates the oca behavour of the objectve functon on the feasbe set by sampng t aong the coordnate drectons. Whenever a sutabe descent feasbe coordnate drecton s detected a new pont s produced by performng a nesearch aong ths drecton. The nformaton progressvey obtaned durng the terates of the agorthm can be used to bud an approxmaton mode of the objectve functon. The mnmum of such a mode s accepted f t produces an mprovement of the objectve functon vaue. We aso derve a bound for the mt accuracy of the agorthm n the mnmzaton of nosy functons. Fnay, we report the resuts of a premnary numerca experence. Keywords: dervatve-free agorthm, bound constrants, nesearch technque 1. Introducton Practca appcatons very often ead to the mnmzaton of a smooth functon whose varabes are subject to bound constrants. In many of these cases the objectve functon vaue s obtaned by drect measurements or t s the resut of a compex system of cacuatons, such as a smuaton. Therefore, even f t s nown that the objectve functon s smooth, ts anaytca expresson s not avaabe and the computaton of ts vaues may be expensve and/or affected by the presence of nose. Hence, the frst order dervatves cannot be expcty cacuated or approxmated. Ths motvates the ncreasng nterest n studyng new dervatve-free methods for bound constraned optmzaton. Such methods shoud present strong goba convergence propertes, shoud be abe to mae sgnfcant progresses wth few functon evauatons, and shoud be suffcenty robust n the nosy case. Wth ths n mnd, n ths paper we consder the probem mnmze f (x) (1) subject to x u, where x,, u R n, wth < u, and f : R n R s a contnuousy dfferentabe functon, but ts frst order dervatves cannot be expcty cacuated or approxmated. We aow the possbty that some of the varabes are unbounded by permttng both = and

2 120 LUCIDI AND SCIANDRONE u = for some {1,...,n}. We denote the feasbe set by F ={x R n : x u}. We defne a statonary pont of probem (1) a feasbe pont x that satsfes the foowng frst-order necessary optmaty condton: f ( x) T (y x) 0 for a y F. (2) We reca that at any statonary pont x of f (x) ( x F), we have that red f ( x) = 0 where the reduced gradent red f (x) s defned as foows max( f (x), 0) f x = u red f (x) = mn( f (x), 0) f x = f (x) otherwse (3) In order to overcome the ac of gradent nformaton, many gobay convergent dervatve-free agorthms proposed n terature are based on the dea of performng fner and fner sampngs of the objectve functon aong sutabe sets of search drectons (see, for nstance, [2, 11, 14 16] and the references quoted there). In [8] t has been performed a genera anayss of the requrements on the search drectons and the sampng technques, whch ensure the goba convergence of a dervatve-free agorthm for unconstraned mnmzaton probems. Roughy speang, at every non-statonary pont, the set of search drectons must contan a descent drecton, and the sampng technque must produce a sutabe pont aong such a drecton. As ceary descrbed n [7], the presence of bound constrants mposes stronger restrctons on the choce of search drectons. In partcuar, at every non-statonary pont, the set of search drectons must contan a descent drecton whch s aso feasbe, n the sense that (suffcenty) sma stepszes aong such drecton must produce feasbe ponts where the objectve functon s reduced. In [7] t s shown that the set of coordnate drectons satsfes ths property. Gobay convergent agorthms usng these drectons have been proposed n [7] and [3]. More n partcuar, the agorthm proposed n [7] foows a pattern search strategy by evauatng the objectve functon on specfed geometrc patterns. Whe the method ntroduced n [3] s based on the dea of usng approxmatng modes of the objectve functon whch are but by usng sutabe grd ponts. In ths paper, we propose a new agorthm mode for sovng probem (1). In order to try to guarantee both goba convergence propertes and a good effcency of the proposed agorthm, we have drawn our nspraton from the strategy underyng the gradent based methods. The goba convergence and the good computatona behavour of these methods foow from the fact that they are abe () to fnd a good feasbe descent drecton (namey a feasbe drecton aong whch the objectve functon suffcenty decreases); () to perform a suffcenty arge stepength aong ths drecton;

3 BOUND CONSTRAINED OPTIMIZATION 121 () to expot the nformaton on the objectve functon obtaned durng the terates of the agorthm. Here, startng from the approaches proposed n [3, 7] and [8], we have defned an agorthm whch tres to foow the ponts () () wthout usng any nformaton on the frst order dervatves and tang account the partcuar structure of the feasbe set. The man features of the agorthm are the foowng: a good feasbe descent drecton s determned by nvestgatng the oca behavour of the objectve functon on the feasbe set aong the coordnate drectons; whenever a sutabe descent feasbe coordnate drecton s detected, a new pont s produced by performng a dervatve free nesearch aong ths drecton; the nformaton progressvey obtaned durng the terates of the agorthm can be used to bud an approxmaton mode of the objectve functon n order to mprove the oca behavour of the agorthm. As regards the theoretca propertes of the proposed agorthm, we prove that every mt pont of the sequence produced s a statonary pont for probem (1). Smary to [1, 3, 6, 17], we consder aso the mnmzaton of nosy functons whch are perturbatons of smooth functons. For ths case, by requrng the standard assumpton that the gradent of the objectve functon s Lpschtz contnuous (whch s not requred n the noseess case), we derve a bound for the mt accuracy of the agorthm. The paper s organzed as foows. In Secton 2 we descrbe the proposed agorthm mode. In Secton 3 we prove the goba convergence of the agorthm. In Secton 4, we characterze the behavour of the agorthm n the mnmzaton of nosy functons. Fnay, n Secton 5 we report the resuts of a premnary numerca experence performed on both standard test probems and a rea appcaton. Notaton. The j-th component of a vector v R n s ndcated by v j. We denote the Eucdean norm (on the approprate space) by. A subsequence of {x } correspondng to an nfnte subset K w be denoted by {x } K. We ndcate by e, wth = 1,...,n, the orthonorma set of the coordnate drectons and by F the boundary of the feasbe set F. Fnay, gven a rea number a, a represents the argest nteger that s not greater than a. 2. A new agorthm mode In ths secton we propose a new cass of dervatve-free agorthms for the mnmzaton of a contnuousy dfferentabe functon n the case that some of (or a) the varabes are bounded. As sad before, the approach s based on the dea of performng sutabe sampngs of the objectve functon aong the coordnate drectons. As ponted out n [7], the coordnate drectons aow us to cope wth the presence of box constrants. Ths can be easy derved from the optmaty condtons (2). In fact, f a feasbe pont x s not a statonary pont of f, then there must exst a feasbe pont y and an nteger h {1,...,n} such that h f ( x) T (y x) h < 0. If ᾱ = (y x) h > 0, then, tang nto account that F s defned by box constrants, we have ᾱ f ( x) T e h < 0, x ᾱe h F

4 122 LUCIDI AND SCIANDRONE The contnuty of f and the convexty of the feasbe set F mpy that there exsts a postve vaue such that: f ( x αe h )< f ( x), x αe h F, for a α (0, ). The case ᾱ = (y x) h < 0 eads to the same concusons wth e h repaced by e h. Hence, n correspondence to any feasbe pont x whch s not a statonary pont, there s a coordnate drecton aong whch (or aong ts opposte) there must exst feasbe ponts where the functon s strcty decreased (ths property s not necessary true for dfferent sets of n neary ndependent drectons). Therefore, performng fnder and fner sampngs of the objectve functon aong the coordnate drectons and ther opposte, t s possbe ether to understand that a pont s a good approxmaton of a statonary pont of f,orto determne a specfc drecton aong whch the objectve functon decreases. On ths bass, we propose an agorthm mode whch sampes the objectve functon aong the coordnate drectons, wth the am of detectng a feasbe drecton where the objectve functon s suffcenty decreased. Once such a drecton has been ndvduated, a dervatvefree nesearch technque s adopted for performng a suffcenty arge step aong t, so as to expot the descent property of the search drecton as much as possbe. Perodcay, t s admtted the possbty of generatng a pont by a movement of arbtrary ength aong an arbtrary drecton. The use of the coordnate drectons as search drectons and the partcuar sampng technque adopted aow us to overcome the ac of gradent nformaton and to ensure that every mt pont of the sequence produced s a statonary pont for probem (1). Formay, the agorthm mode s descrbed as foows. Agorthm Mode Data. x 0 F, (0, 1), γ > 0, 0 < 0 <, d = e for = 1,...,n. Step 0. Set = 0, = 1, h = 1. Step 1. Compute α max s.t. x α max d F and set α = mn{,α max}. If α>0and f (x αd ) f (x ) γ(α) 2 go to Step 3. Step 2. Compute α max s.t. x α max d F and set α = mn{,α max}. If α>0and f (x αd ) f (x ) γ(α) 2 then set d = d and go to Step 3. ese set α = 0, 1 = α, and go to Step 4. Step 3. Compute α by the Expanson Step(d,α,α max,γ)and set 1 = α. Step 4. Set x 1 = x α d, j 1 = j, for j {1,...,n} and j. Step 5. If h n then fnd x 1 such that f (x 1 ) f ( x 1 ) and x 1 F, (4) ese set x 1 = x 1. If x 1 x 1 then set h 1 = 1 ese set h 1 = h 1. Set = mod(, n) 1, = 1 and go to Step 1.

5 BOUND CONSTRAINED OPTIMIZATION 123 Expanson Step (d,α,α max,γ). Data. δ (0, 1). Step 1. Let = mn{α max, α δ }. If α = α max or f (x d )> f (x ) γ 2 set α = α and stop. Step 2. Set α = and go to Step 1. More n partcuar, the steps of the agorthm can be summarzed as foows. At Step 1 the drecton d s examned wth the am of determnng (f possbe) a feasbe pont where the objectve functon s suffcenty decreased. Frst, t s computed the maxmum feasbe stepength α max whch can be performed aong the drecton d startng from the pont x. Then, the tra stepsze α s determned by choosng the mnmum between α max and. The scaar has been computed on the bass of the behavour of the objectve functon aong the same drecton showed at the prevous teratons. Therefore, the scaar shoud tae nto account the senstvty of the objectve functon wth respect to the -th varabe, and hence t shoud provde a promsng nta stepsze for the drecton d. Fnay, t s verfed f the movng of ength α aong d produces a feasbe pont where the functon s suffcenty reduced. If such a pont s produced then a nesearch technque s performed aong d to provde a sutabe stepsze α (Step 3). Otherwse, the drecton d s consdered (Step 2). Step 2 s smar to Step 1, wth d repaced by d. In ths case, f the tra pont x αd does not produce a suffcent decrease of f then the stepsze α s set equa to zero and the scaar s reduced. In ths way, when the drectons d and d w be consdered agan by the agorthm, the nta stepsze w be chosen n an nterva contanng smaer vaues. At Step 3 a sutabe arge stepsze α s computed by a dervatve-free nesearch technque. Such technque derves from the ones proposed n [4]. It computes a suffcenty good estmate of the mnmum of f aong d wthout requrng any nformaton on the sope of the objectve functon. The am of ths step s to expot the good descent drecton d dentfed at Step 1 or Step 2. Then, the scaar 1 s set equa to α. The motvaton of ths choce derves from the fact that the stepsze α produced by a nesearch technque shoud dentfy promsng vaues for the nta stepsze when the drecton d (or d ) w be nvestgated. At Step 4 the canddate pont x 1 s generated. At Step 5 the new pont x 1 s produced and, for the next teraton, a coordnate drecton s seected by foowng the cycc order. At each teraton x 1 can be aways set equa to the canddate pont x 1 produced at Step 5. The ndex h counts the number of successve teratons characterzed by the fact that x 1 = x 1. The condton h n ndcates that the ast n ponts generated have been produced by nvestgatng the behavour of f aong a the coordnate drectons and that, roughy speang, the agorthm has obtaned enough nformaton about the oca behavour of the objectve functon. In ths case, the agorthm admts the possbty of acceptng as x 1 any feasbe pont whch produces a reducton of the objectve functon. Therefore, n these teratons the pont x 1 can be generated, for nstance, by mnmzng any approxmaton mode of the objectve functon but by usng the nformaton progressvey obtaned by the agorthm. Ths

6 124 LUCIDI AND SCIANDRONE possbty does not affect the convergence propertes of the agorthm, but t can ncrease ts effcency. 3. Convergence anayss In ths secton we study the theoretca propertes of Agorthm Mode. In partcuar, we show that any accumuaton pont of the sequence generated by the proposed agorthm s a statonary pont of probem (1). Frst, we state the foowng proposton. Proposton 1. Suppose that f s bounded beow on the feasbe set F and et {x } be the sequence produced by Agorthm Mode. Then: () Agorthm Mode s we defned; () every mt pont of {x } beongs to F; () we have m = 0 (5) m = 0 for = 1,...,n. (6) Proof: In order to prove that Agorthm Mode s we defned, we have to ensure that the Expanson Step, when performed aong a drecton d, wth {1,...,n}, termnates n a fnte number j of steps. At ths am, by contradcton we assume that for a gven d and x δ j αd F for a j, f (x δ j αd )< f (x ) γ(δ j α) 2 d 2 for a j, whch voates the assumpton that f s bounded beow on F. As regards asserton (), we have that the nstructons of Agorthm Mode mpy that x F for a. Snce F s a cosed set, the asserton s proved. In order to prove (5), we spt the teraton sequence {} nto two parts, K and K.We dentfy wth K those teratons where α = 0 (7) and wth K those teratons where α 0 s produced by Expanson Step. Then, Steps 3 and 5 mpy f (x 1 ) f (x α d ) f (x ) γ(α ) 2 d 2. (8) Tang nto account the boundedness assumpton on f, t foows from (8) that { f (x )} tends to a mt f.ifk s an nfnte subset, recang that d =1 we obtan m, K α = 0. (9) Therefore, (7) and (9) mpy (5).

7 BOUND CONSTRAINED OPTIMIZATION 125 In order to prove (6), for each {1,...,n} we spt the teraton sequence {} nto three parts, K 1, K 2 and K 3. We dentfy wth K 1 those teratons where Expanson Step has been performed usng the drecton d, namey f (x α d ) f (x ) γ(α ) 2 d 2 (10) 1 = α. (11) We denote by K 2 those teratons where we have faed n decreasng the objectve functon aong the drectons d and d. By the nstructons of the agorthm t foows that for a K 2 1, (12) where (0, 1). Fnay, K 3 denotes the teratons where the drectons d and d are not used as search drectons. Then, for K 3 we have 1 =. (13) If K 1 s an nfnte subset, from (11) and (5) we get that m 1 = 0. (14), K 1 Now, et us assume that K 2 s an nfnte subset. For each K 2, et m (we omt the dependence from ) be the bggest ndex such that m < and m K 1. Then we have: 1 (1 m ) n m m (15) (we can assume m = 0 f the ndex m does not exst, that s, K 1 s empty). As and K 2, ether K 1 s an nfnte subset mpyng m,ork 1 s fnte mpyng ( 1 m ). Hence, f K 2 s an nfnte subset, (15) together wth (14), or the fact that (0, 1), yeds m 1 = 0. (16), K 2 Fnay, et us consder the nfnte subset K 3 (note that the nstructons of the agorthm mpy that K 3 s aways an nfnte set). The nstructons of the agorthm mpy that, for a K 3 and suffcenty arge, there exsts a nonnegatve ndex ν n such that ν K 1 K 2 1 = ν 1. Therefore, from (14) and (16), we get that m 1 = 0, (17), K 3 so that (6) s proved, and ths concudes the proof.

8 126 LUCIDI AND SCIANDRONE Now we are ready to prove the man convergence resut. Proposton 2. Suppose that f s bounded beow on the feasbe set F and et {x } be the sequence produced by Agorthm Mode. Then every mt pont of {x } s a statonary pont for probem (1). Proof: Let x be any mt pont of {x }, that s m x = x, (18), K where K {0, 1,...}. By () of Proposton 1 we have that x F. In order to prove the thess, et us suppose by contradcton that x s not a statonary pont. Therefore, there exsts a pont ȳ F such that f ( x) T (ȳ x) <0. (19) Ths mpes that h f ( x)(ȳ x) h < 0, (20) for some h {1,...,n}. Let us defne the sequence of scaars β = (ȳ x ) h. Now, by (20) we have where m, K β f (x ) T e h = β f ( x) T e h < 0, (21) β = (ȳ x) h > 0. (22) For each K, et us consder the smaest ndex m and the bggest ndex m such that n the teratons m and m the drecton e h or/and the drecton e h are nvestgated. The nstructons mpy m n m n. (23) Moreover we have that at east one of the foowng occurrence must happen () a the ponts x j, wth 0 j (m ), are produced by Step 4, that s x j1 = x j α j e ( j) (or x j1 = x j α j e ( j) ); () a the ponts x j, wth 0 j m, are produced by Step 4, that s x j1 = x j α j e ( j) (or x j1 = x j α j e ( j) ),

9 BOUND CONSTRAINED OPTIMIZATION 127 where the ndex ( j) {1,...,n} dentfes the search drecton nvestgated. Now et K K and K K be the subsets such that condton () s verfed for a K and condton () s verfed for a K. Suppose frst that K s nfnte. For sempcty we rename K as K. Hence, () of Proposton 1 mpes that, for 1 j m, t foows m, K x j x j 1 =0, (24) from whch, recang (23), we get m, K x m x = x. (25) By (21), (22) and (25) we obtan m, K β m = β. (26) Condtons (21), (25) and (26) mpy that for K and suffcenty arge the drecton sgn(β m )e h s an ascent drecton n x m because t maes an acute ange wth the gradent, and moreover, recang (6) we can prove that for K and suffcenty arge f ( x m sgn ( β m ) h m 1 In fact, by appyng the Mean Vaue Theorem we have f ( x m sgn ( β m = sgn ( β m ) h m 1 ) e h > f ( ) x m. (27) ) e h f ( ) x m ) h m 1 [ f ( x) T e h ( f (ξ ) T e h f ( x) T e h )], ) h m where ξ = x m λ sgn(β m 1 e h, wth λ (0, 1). Then, snce ξ x m 0, recang (21), (26) and the contnuty of f, t foows that (27) hods. Hence, for suffcenty arge, at the m -th teraton, the nstructons of the agorthm (n partcuar, Step 2) mpy that the drecton sgn(β m )e h s nvestgated. Snce x F and h ȳ h u h t foows x m tsgn ( ) β m eh = x m t β m (ȳ xm ) heh F t [ 0, β m ]. (28) Now, by () of Proposton 1 we have that α 0 and h m for K and suffcenty arge, (22), (26) and (28) yed 0 for, and hence, x m sgn ( β m ) h m 1 e h F (29)

10 128 LUCIDI AND SCIANDRONE and x m sgn ( )α h m β m δ e h F (30) Therefore we have that ether or α m = 0,(29) hods and ( f x m sgn ( β m ) h m 1 ) e h > f ( ( ) h ) 2 m x m γ. (31) α m 0,(30) hods and ( f x m sgn ( ) )α m β m δ e h > f ( x m By appyng the Mean Vaue Theorem n (31) and (32) we can wrte ether ( ) ) αm 2 γ. (32) δ or sgn ( β m ) ( ) T h m f um eh 1 where sgn ( β m u m v m = x m = x m ) ( ) T f vm eh > α m δ sgn ( β m sgn ( β m ) h λ 1 m 1 e h, ) α λ 2 m δ e h, wth λ 1,λ2 (0, 1). Therefore, tang the mts for and K, by usng (5), (6) and the contnuty assumpton on f, t foows sgn( β) f ( x) T e h 0, whch contradcts (21). Now, f K s an nfnte set then we can repeat the same reasonngs by mnor modfcatons and obtan a contradcton wth (21). Remar 1. If we repace at Step 5 of Agorthm Mode condton (4) wth the foowng stronger condton f (x 1 ) f ( x 1 ) and x 1 F (33) x 1 x max{ f,ρα } (34)

11 BOUND CONSTRAINED OPTIMIZATION 129 where f = f (x 1 ) f (x ) and ρ (0, 1), we obtan that the resuts of Proposton 1 and Proposton 2 st hod wth the addtona property of the sequence {x }: m x 1 x =0. (35) In fact, snce f s bounded beow on F and the sequence { f (x )} s decreasng, we have that f 0. On the other hand, by (5) of Proposton 1 we have that α 0. Therefore, (35) foows mmedatey. 4. Convergence anayss n presence of nose In ths secton, we consder the case where the vaues of the objectve functon of probem (1) are corrupted by the presence of nose. In other words, we can ony observe the perturbaton f (x) gven by f (x) = f (x) (x), where (x) represents the amount of nose. Furthermore, we assume that there exsts a constant >0 such that (x) for every x F. (36) In Agorthm Mode the vaue f (x) has to be repaced by f (x). Frst of a, we remar that t s possbe to show that Proposton 1 st hod aso n presence of nose satsfyng (36). Further resuts about the propertes of the proposed agorthm can be stated under the assumpton that the gradent of the objectve functon s Lpschtz contnuous. In partcuar, t s possbe to derve a bound on the norm of the reduced gradent. At ths am, for each teraton we consder a ba B of radus r = 2 mn{,δ} n (37) =1 about x (where and δ are user chosen parameters of the agorthm). For each teraton such that h n we have that every pont where the functon has been evauated n one of the precedng n teratons s contaned n the ba B about x. In fact, as regards the generated ponts we have x x n n α 1 =1 n. =1 Then, n correspondence to each teraton j, the agorthm can sampe the objectve functon aong a gven coordnate drecton e h n the nterva [ ] x j h j1 mn{,δ} e h, x j h j1 mn{,δ} e h. Then, we have the foowng proposton.

12 130 LUCIDI AND SCIANDRONE Proposton 3. Suppose that f s bounded beow on the feasbe set F and that ts gradent s Lpschtz contnuous (wth constant L) on F. Let {x } be the sequence produced by Agorthm Mode and et = sup x B (x). Then, there exst two constants c 1, c 2 > 0 such that, for each teraton for whch h n, we have red f (x ) c 1 max =1,...,n { } c2 mn =1,...,n { }. Proof: Let us consder the -th component of red f (x ). Now et () be the bggest ndex such that at the () teraton the drecton e (and/or e ) has been nvestgated. Frst of a, we note that by the nstructons of the agorthm, and recang the assumpton h n, t foows that () n, and by the defntons of B and we have that the objectve functon vaues computed n the teratons (), () 1,..., are corrupted by an amount of nose bounded by. Now we dstngush the foowng cases: () x = u () x = () < x < u Case (). In ths case we have ether (a) x () = u or (b) x () < u. In case (a), as x = x (), from the nstructons of the agorthm we have α () = 0, ()1 =, and ( ) f x () e > f ( ( ) ) 2 x () γ 2. By appyng the Mean Vaue Theorem we obtan f ( ( ) T ) 2 u () e > γ 2, where u () = x () λ () e wth λ () (0, 1). Then, we can wrte [ ( ) f u() f (x ) f (x ) ] T e <γ 2,

13 BOUND CONSTRAINED OPTIMIZATION 131 from whch, tang nto account the Lpschtz assumpton on f, t foows f (x ) T e <γ 2 L x u () γ 2 L x x () L. (38) We have () 1 x = x () α () j d () j, j=0 where d () j {e 1, e 1,...,e n, e n }. For each j such that α () j 0, recang the nstructons of the agorthm, we have that there exsts an ndex {1,...,n} such that () j1 = α () j, and = () j1. Therefore, t foows { } α () j max, =1,...,n and we can wrte { } x x () n max. =1,...,n From (38) we get red (γ L(n 1)) { } f (x )< max 2 { }. (39) =1,...,n mn =1,...,n In case (b), we have α () 0, ()1 = α () =, and f ( x () e ) f ( x() ) γ ( ) 2 2 f ( x () ) γ ( ) 2 2. Then, by appyng the Mean Vaue Theorem, we obtan f ( ) T v () e <γ 2. where v () = x () λ () e wth λ () (0, 1). Then, by repeatng the precedng reasonngs we obtan red { } f (x )<(γ L(n 1)) max 2 { } (40) =1,...,n mn =1,...,n Case (). It s anaogous to Case (), so that condtons (39) and (40) hod.

14 132 LUCIDI AND SCIANDRONE Case (). In ths case we have ether (a) x () = x or (b) x () x. In case (a), from the nstructons of the agorthm, recang that ()1 = = α, where α s the nta stepsze, we have ( ) f x () e f ( ( ) ) 2 x () γ 2 (41) ( ) f x () e f ( ( ) ) 2 x () γ 2. (42) By appyng the Mean Vaue Theorem we obtan f ( ( ) T ) 2 u () e γ 2 (43) f ( ( ) T ) 2 v () e γ 2, (44) where u () = x () λ() 1 e,v () = x () λ() 2 e, wth λ() 1,λ2 () (0, 1). From (43), tang nto account the Lpschtz assumpton on f we get f (x ) T e γ 2 L x u () γ 2 L x x () L γ 2 { } nl max L =1,...,n = (γ L) 2 { } nl max. =1,...,n Hence, t foows red (γ L(n 1)) { } f (x ) max 2 { }. (45) =1,...,n mn =1,...,n From (44), by repeatng the same reasonngs, we obtan red (γ L(n 1)) { } f (x ) max 2 { }. (46) =1,...,n mn =1,...,n

15 BOUND CONSTRAINED OPTIMIZATION 133 From (45) and (46) t foows red f (x ) (γ L(n 1)) { } max 2 { }. (47) =1,...,n mn =1,...,n Let us consder the case (b). Wthout oss of generaty, we can assume that n ths case we have α () 0, ()1 = = α (), and wth f ( x () e ) f ( x() ) γ ( ) 2 2 f ( x () ᾱ e ) > f ( x() ) γ (ᾱ ) 2 2, ᾱ = δ () where δ () = δ f x () δ e F, and δ () (δ, 1) otherwse (n ths case δ () s such that x () δ () e F ). By appyng the Mean Vaue Theorem, we can wrte and f ( u () ) T e γ 2 γ 2 α, f ( ) T v () e > γ 2 δ (), δ () where u () = x () λ() 1 α ()e,v () = x () λ() 2 δ () e wth λ() 1,λ2 () (0, 1). Tang nto account the Lpschtz assumpton on f we can wrte f (x ) T e γ 2 γ 2 L x u () L x x () L ()1 (γ L) 2 { } nl max, =1,...,n f (x ) T e > γ 2 δ () δ () L x v () γ δ 2 L x x () L (γ L) δ 2 { } nl max. =1,...,n

16 134 LUCIDI AND SCIANDRONE Then we have red f (x ) (γ L(n 1)) { } max 2 δ { } (48) =1,...,n mn =1,...,n Fnay, from (39), (40) and (48) we obtan ( ) red f (x ) n 1/2 (γ L(n 1)) { } max 2 max{,δ} { }, =1,...,n mn =1,...,n and ths concudes the proof. Remar 2. We note that f at each teraton we set x 1 = x 1 (namey x 1 = x α d, where d {e 1,...,e n, e 1,..., e n } and where α s equa to zero or s produced by the Expanson Step) then the asserton of Proposton 3 hods for a. In fact, n ths case we have h n for a > n. Fnay, by requrng a stronger assumpton on the nose, we have the foowng convergence resut. Proposton 4. Suppose that f s bounded beow on the feasbe set F and that ts gradent s Lpschtz contnuous (wth constant L) on F. Let {x } be the sequence produced by Agorthm Mode. Then f m mn =1,...,n { } = 0, (49) then every mt pont of {x } s a statonary pont for probem (1). Proof: Let x be any mt pont of {x }, that s m, K x = x, where K {0, 1,...}. In order to prove the thess, by contradcton, et us suppose that x s not a statonary pont for probem (1), that s red f ( x) ɛ, (50) for some ɛ>0. From the nstructons of the agorthm we have that, for each K, there exsts an ndex ν, wth 0 ν < n such that h ν n and x j = x j 1 α j 1 e ( j) for j = 1,...,ν. (51)

17 BOUND CONSTRAINED OPTIMIZATION 135 By Proposton 3 we have red f (x ν ) c 1 max =1,...,n { ν } c2 ν mn =1,...,n { ν }, hence, recang (49) and that pont () of Proposton 1 st hods aso n presence of bounded nose, t foows m, K red f (x ν ) =0. (52) Then, by usng the Lpschtz assumpton on f, we can wrte ν red f (x ) red f (x ν ) L x x ν L α j 1, from whch, recang agan () of Proposton 1, the contnuty assumpton of f and (52) we obtan red f ( x) =0, j=1 whch contradcts (50). 5. Premnary computatona resuts In order to evauate a possbe practca nterest of the proposed agorthm mode, we have used 41 standard test probems of dmenson n ranng form 2 to 10. These probems are seected from two test sets: the frst one s made of 23 probems defned n [3], whch are obtaned from the set of functons suggested n [10]. The second set conssts of the a box constraned probems of the Hoc-Schttows coectons [5, 12]. Furthermore, we have consdered a rea test probem whch derves from an appcaton [13] regardng the desgn of nstruments for magnetc resonance. On the bass of the proposed agorthm mode, we have mpemented n Fortran code two dfferent agorthms. Agorthm 1. The pont x 1 produced at Step 5 s set equa to x 1, therefore, ether x 1 = x or x 1 = x α e (x α e ), where {1,...,n}. Agorthm 2. It tres to produce the pont x 1 at Step 5 by usng an approxmaton scheme. Now we descrbe more n deta the choces made n these mpementatons Choce of parameters The parameters whch appear n the agorthm mode have been set as foows. γ = 10 6 δ = 0.25 = = 0.5 = 1,...,n.

18 136 LUCIDI AND SCIANDRONE We note that we have not performed an extensve emprca tunng of the parameters of the agorthm. We have adapted choces usuay adopted n nesearch technques of gradentbased agorthms Approxmaton scheme used n Agorthm 2 In Agorthm 2 we have made an nta attempt to expot the nformaton on the objectve functon obtaned at the prevous teratons n order to produce a better pont x 1 wth respect to x 1. In partcuar, every n teratons, we construct (when the agorthm has produced a suffcent number of ponts cose to x ) a smpe quadratc mode q(x) = 1 2 x T Qx c T x b of the objectve functon. The N = n(n 1)/2 n 1 free parameters of the mode are determned by mnmzng the error M (q(x j ) f (x j )) 2 (53) j=1 beng M > 0 and x 1,...,x M the ast M ponts where f has been evauated and such that x j ũ, wth = max {, x 100 } ũ = max { u, x 100 } = 1,...,n = 1,...,n. Then, we appy a mnmzaton method for computng a statonary pont of the defned quadratc probem mn q(x) (54) x ũ and the obtaned pont s accepted f t produces a reducton of f. We note that the box [, ũ] has the roe of seectng ponts whch are not too far from the current pont x. In our tests we have chosen M = N 5 and we have used the routnes F04JAF and E04NAF of NAG brary to sove, respectvey, the near east squares probem (53) and the box constraned quadratc probem (54) Stoppng crteron Snce the defnton of an effcent stoppng crteron s out of the scope of the wor, we have adapted the same approach proposed n [3]. Let f 0 be the vaue of f at the startng

19 BOUND CONSTRAINED OPTIMIZATION 137 pont x 0 and f be the best nown functon vaue. Then we ntroduce the quotent q = f (x ) f f 0 f, whch can be consdered a measure of the speed of convergence and we have stopped an agorthm whenever q ɛ, (55) where ɛ>0saprefxed vaue. By usng dfferent vaues of ɛ we can have an dea on the effcency of an agorthm. However, n some test probems the goba mnmum s not the unque statonary pont. Therefore, an agorthm coud generate a sequence convergng towards a statonary pont x, wth f ( x) > f, and coud never satsfy the crteron (55). To tace ths possbe occurrence, we have ntroduced aso the foowng stoppng crteron max =1,...,n { } (56) Fnay, we say that an agorthm has faed when t has performed a number N max = 1000 of functon evauatons wthout satsfyng any of the two the stoppng crtera Numerca experence We have tested Agorthms 1 and 2 on the set of standard probems wth three dfferent vaues of ɛ n crteron (55), namey ɛ = 10 1, ɛ = 10 3, ɛ = The compete resuts are reported n [9]. Here we descrbe some summares of these resuts. In the foowng tabe we report the tota number of functon evauatons needed to sove the probems where both the agorthms have been abe to satsfy crteron (55), the tota number of faures and the tota number of stops due to crteron (56). From the resuts of Tabe 1 we can note that for ɛ = 10 1 both the agorthms have been abe to sove a the test probems. For ɛ = 10 3 most of the test probems are st soved by the two agorthms. Whereas the case ɛ = 10 6 ponts out the utty of approxmaton technques n a dervatve-free agorthm. In fact, when the degree of requred precson s hgh, t seems to be necessary to expot as much as possbe the nformaton on the Tabe 1. Cumuatve resuts. Tota n f No. of faures No. of stops ɛ = 10 1 Ag Ag ɛ = 10 3 Ag Ag ɛ = 10 6 Ag Ag

20 138 LUCIDI AND SCIANDRONE Tabe 2. Number of WINS. No. of wns n terms of n f No. of baances ɛ = 10 1 Ag Ag ɛ = 10 3 Ag Ag ɛ = 10 6 Ag Ag objectve functon obtaned durng the teratons of the agorthm. From Tabe 1 we can aso note that the smpe approxmaton scheme used n Agorthm 2 has aowed us a sgnfcant computatona savng n terms of number of functon evauatons. Then we have anaysed more n deta the behavour of Agorthms 1 and 2. In partcuar, we say that an agorthm wns f the number of functon evauatons requred to sove a test probem s smaer or equa to the 95% of the number requred by the other agorthm. In Tabe 2 we report the tota number of wns. Tabe 2 shows that n many soved probems the performances of the two agorthms are comparabe and hence, that Agorthm 2 outperforms Agorthm 1 ony n few probems. However, tang nto account aso the resuts of Tabe 1, we can concude that n such probems the mprovement due to the use of an approxmaton s consderabe. In order to have a frst dea on the practca nterest of the proposed approach, we have compared Agorthm 2 wth a method usng fnte-dfferences gradents. As t s we nown, methods of ths cass are very effcent n absence of nose. Therefore, the comparson s an hard test for Agorthm 2 whch s a drect search method, namey a method whch does not try to approxmate expcty the frst order dervatves. In partcuar, we have used E04UCF routne of NAG brary, whch s a sequenta quadratc programmng method. Smary to Agorthm 2 we have stopped the NAG routne whenever (55) has been satsfed (aso n ths case we have used three dfferent vaues of ɛ:10 1, 10 3, 10 6 ). In some runs E04UCF routne was termnated by ts stoppng crteron and the vector returned x was a statonary pont such that f ( x) > f. In two test probems, E04UCF was unabe to produce a pont dfferent from the startng pont. We have consdered these cases as faures of E04UCF. The compete resuts regardng E04UCF are reported n [9]. Here, for brevty s sae, we report a summary of the comparsons between Agorthm 2 and E04UCF (n Tabes 3 and 4). In partcuar, n Tabe 3 we report the tota number of functon evauatons needed to sove the probems where both the Agorthm 2 and the consdered NAG routne have been abe to satsfy crteron (55), the tota number of faures and the tota number of stops due to crteron (56) for Agorthm 2, and to the defaut stoppng crteron for the NAG routne. In Tabe 4 we show the tota number of wns n terms of number of functon evauatons. From the resuts of Tabe 3, we can note that for ɛ = 10 1 Agorthm 2 outperforms E04UCF. For ɛ = 10 3 t s st compettve n terms of number of functon evauatons wth E04UCF and the two agorthms are comparabe n terms of faures. When ɛ = 10 6, the

21 BOUND CONSTRAINED OPTIMIZATION 139 Tabe 3. Cumuatve resuts. Tota n f No. of faures No. of stops ɛ = 10 1 Ag E04UCF ɛ = 10 3 Ag E04UCF ɛ = 10 6 Ag E04UCF Tabe 4. Number of WINS. No. of wns n terms of n f No. of baances ɛ = 10 1 Ag E04UCF 8 1 ɛ = 10 3 Ag E04UCF 10 1 ɛ = 10 6 Ag E04UCF 11 0 behavour of Agorthm 2 s comparabe wth the one of E04UCF n terms of number of functon evauatons. However, E04UCF s abe to sove a (sghty) arger number of probems. The precedng resuts, athough far from beng exhaustve, show a satsfactory behavour of Agorthm 2 compared wth a method usng fnte-dfferences dervatves. We reca that methods of ths nd are effcent n the case that the objectve functon vaues are not affected by any nose, but they are not consdered reabe for sovng nosy probems. In fact, the behavour of methods usng fnte-dfferences dervatves deterorates aso wth a sma nose. In partcuar, we have repeated the tests by consderng functon vaues gven by f (x) = f (x)(1 η), η N(0,σ 2 ), where N(0,σ 2 ) denotes a Gaussan dstrbuted random number wth zero mean and varance σ 2 = We note that the convergence anayss deveoped n Secton 4 hods for bounded nose, whe the one consdered s unbounded. However, due to the sma vaue of the varance, the nose can be consdered bounded n practce. We have obtaned the foowng resuts. Agorthm 2: for the three vaues of ɛ, the number of faures s not changed wth respect to the noseess case. E04UCF routne: for ɛ = 10 1 the number of faures changes from 2 to 27; for ɛ = 10 3 the number of faures changes from 2 to 32; for ɛ = 10 6 the number of faures changes from 2 to 32.

22 140 LUCIDI AND SCIANDRONE 5.5. A rea desgn probem We consder a rea probem arsng from an appcaton [13] whch regards the optma desgn of apparat for magnetc resonance. The appcaton deas wth the constructon of apparat wth reduced szes to be used for cnca anayss of perphera regons of the body. These apparat are based on resstve magnets that can be manufactured at very ow cost. However, they requre a good magnetc fed unformty n a arge part of ther voume, and the reduced szes mae t dffcut to obtan ths fed unformty. In partcuar, the magnetc fed s generated by four currents: I 1, I 2, I 3, I 4. The requred fed s B 0 and t must be as unform as possbe n a spherca regon at the center of the magnet. The vector of currents s denoted by I = (I 1, I 2, I 3, I 4 ) T and the z component of the magnetc fed generated n a pont r j by B z (I ; r j ). More formay, the probem s mn f (I ) = N p [B z (I ; r j ) B 0 ] 2 (57) s.t. 0 I U j=1 The ower bounds (0 I ) are mposed because the currents vaues must have the same sgn for reasons reated to the constructon of the magnet. The upper bounds (I U) depend on the aowed power dsspaton. We note that, for each r j, wth j = 1,...,N p, the functon B z (.; r j ) : R 4 R s not nown anaytcay, but for each I R 4 the vaue B z (I ; r J ) can be drecty measured by a Gaussmeter. However, the evauaton of the objectve functon s not expensve. In [13], neura modes have been defned for approxmatng the functons B z (.; r j ), wth j = 1,...,N p, by usng a massve data set generated offne. Then, a gradent-based method has been apped to probem (57), where the terms B z (I ; r j ) have been repaced by the anaytca neura mode determned. In practce some parameters characterzng the magnet w change over the tme, and hence, the fed unformty obtaned by the procedure proposed n [13] can become unacceptabe. In ths case, an onne procedure s necessary for recomputng qucy the vector of currents startng from the currents prevousy determned offne. Ths procedure shoud be abe to sove probem (57) by usng drect measurements for computng the vaues B z (I ; r J ). Snce the objectve functon vaues w be affected by an amount of nose (due to the drect measurements), the use of a method based on fnte-dfferences gradents w be mpractca, whe the adopton of a dervatve-free method appears sutabe. In order to evauate the potentates of ths approach, we have apped Agorthm 2 to probem (57), where for each feasbe vector I, the vaues B z (I ; r j ) are determned by a smuaton program. In partcuar, the stoppng crteron (55) s we suted for ths cass of appcatons. In fact, the vaue f can correspond to an acceptabe vaue of the fed unformty. Startng from a vector of nta currents such that f 0 = , assumng f = 0.049, we have used three dfferent vaues of ɛ n the stoppng crteron, and we have obtaned the foowng resuts (where and n f are, respectvey, the number of teratons and the number of functon evauatons requred to satsfy the stoppng crteron (55), whe f s the objectve functon attaned).

23 BOUND CONSTRAINED OPTIMIZATION 141 ɛ = 10 1 = 32 n f = 102 f = ɛ = 10 3 = 37 n f = 120 f = ɛ = 10 6 = 377 n f = 1060 f = To examne more n deta the behavour of Agorthm 2, we aso report some ntermedate resuts. The resuts show the practcabty of usng Agorthm 2 to update the current vaues n f f In fact Agorthm 2 has been abe to reobtan a good unform fed by performng a mted number of functon evauatons. References 1. E.J. Anderson and M.C. Ferrs, A drect search agorthm for optmzaton wth nosy functons evauatons, Mathematca Programmng Technca Report 96-11, Computer Scences Department, Unversty of Wsconsn, Madson, Wsconsn, A.R. Conn, K. Schenberg, and Ph.L. Tont, Recent progress n unconstraned nonnear optmzaton wthout dervatves, Mathematca Programmng, vo. 79, pp , C. Ester and A. Neumaer, A grd agorthm for bound constraned optmzaton of nosy functons, IMA Journa of Numerca Anayss, vo. 15, pp , L. Grppo, F. Lampareo, and S. Lucd, Goba convergence and stabzaton of unconstraned mnmzaton methods wthout dervatves, Journa of Optmzaton Theory and Appcatons, vo. 56, no. 3, pp , W. Hoc and K. Schttows, Test Exampes for Nonnear Programmng Codes, Sprnger-Verag: Bern, Lectures Notes n Economcs and Mathematca Systems, vo C.T. Keey, Iteratve Methods for Optmzaton. Fronters n Apped Mathematcs, Socety for Industra and Apped Mathematcs: Phadepha, R.M. Lews and V. Torczon, Pattern search methods for bound constraned mnmzaton, SIAM Journa on Optmzaton, vo. 9, pp , S. Lucd and M. Scandrone, On the goba convergence of dervatve free methods for unconstraned optmzaton, Technca Report 18-96, DIS, Unverstà d Roma La Sapenza. 9. S. Lucd and M. Scandrone, A dervatve-free agorthm for bound constraned optmzaton, R , IASI, Consgo Nazonae dee Rcerche. 10. J.J. Moré, B.S. Garbow, and K.E. Hstrom, Testng unconstraned optmzaton software; ACM Trans. on Math. Software, vo. 7, pp , M.J.D. Powe, Drect search agorthms for optmzaton cacuatons, Acta Numerca, vo. 7, pp , K. Schttows, More Test Exampes for Nonnear Programmng Codes. Sprnger-Verag: Bern, Lectures Notes n Economcs and Mathematca Systems, vo M. Scandrone, G. Pacd, L. Testa, and A. Sotgu, Compact ow fed magnetc resonance magng magnet: Desgn and optmzaton, Revew of Scentfc Instruments, vo. 71, no. 3, pp , V. Torczon, On the convergence of pattern search methods, SIAM Journa on Optmzaton, vo. 7, pp. 1 25, 1997.

24 142 LUCIDI AND SCIANDRONE 15. P. Tseng, Fortfed-descent smpca search method: A genera approach, SIAM Journa on Optmzaton, vo. 10, pp , M.H. Wrght, Drect search methods: Once scorned, now respectabe, Numerca Anayss 1995, Proceedngs of the 1995 Dundee Benna Conference n Numerca Anayss, D.F. Grffths and G.A. Watson (Eds.) Addson-Wesey Longman: Harow, UK, 1996, pp S.K. Zavrev, On the goba optmzaton propertes of fnte-dfferences oca descent agorthms, Journa of Goba Optmzaton, vo. 3, pp , 1993.

Research on Complex Networks Control Based on Fuzzy Integral Sliding Theory

Research on Complex Networks Control Based on Fuzzy Integral Sliding Theory Advanced Scence and Technoogy Letters Vo.83 (ISA 205), pp.60-65 http://dx.do.org/0.4257/ast.205.83.2 Research on Compex etworks Contro Based on Fuzzy Integra Sdng Theory Dongsheng Yang, Bngqng L, 2, He

More information

ON AUTOMATIC CONTINUITY OF DERIVATIONS FOR BANACH ALGEBRAS WITH INVOLUTION

ON AUTOMATIC CONTINUITY OF DERIVATIONS FOR BANACH ALGEBRAS WITH INVOLUTION European Journa of Mathematcs and Computer Scence Vo. No. 1, 2017 ON AUTOMATC CONTNUTY OF DERVATONS FOR BANACH ALGEBRAS WTH NVOLUTON Mohamed BELAM & Youssef T DL MATC Laboratory Hassan Unversty MORO CCO

More information

MARKOV CHAIN AND HIDDEN MARKOV MODEL

MARKOV CHAIN AND HIDDEN MARKOV MODEL MARKOV CHAIN AND HIDDEN MARKOV MODEL JIAN ZHANG JIANZHAN@STAT.PURDUE.EDU Markov chan and hdden Markov mode are probaby the smpest modes whch can be used to mode sequenta data,.e. data sampes whch are not

More information

Nested case-control and case-cohort studies

Nested case-control and case-cohort studies Outne: Nested case-contro and case-cohort studes Ørnuf Borgan Department of Mathematcs Unversty of Oso NORBIS course Unversty of Oso 4-8 December 217 1 Radaton and breast cancer data Nested case contro

More information

New Inexact Parallel Variable Distribution Algorithms

New Inexact Parallel Variable Distribution Algorithms Computatona Optmzaton and Appcatons 7, 165 18 1997) c 1997 Kuwer Academc Pubshers. Manufactured n The Netherands. New Inexact Parae Varabe Dstrbuton Agorthms MICHAEL V. SOLODOV soodov@mpa.br Insttuto de

More information

A finite difference method for heat equation in the unbounded domain

A finite difference method for heat equation in the unbounded domain Internatona Conerence on Advanced ectronc Scence and Technoogy (AST 6) A nte derence method or heat equaton n the unbounded doman a Quan Zheng and Xn Zhao Coege o Scence North Chna nversty o Technoogy

More information

A General Column Generation Algorithm Applied to System Reliability Optimization Problems

A General Column Generation Algorithm Applied to System Reliability Optimization Problems A Genera Coumn Generaton Agorthm Apped to System Reabty Optmzaton Probems Lea Za, Davd W. Cot, Department of Industra and Systems Engneerng, Rutgers Unversty, Pscataway, J 08854, USA Abstract A genera

More information

On the convergence of the block nonlinear Gauss Seidel method under convex constraints

On the convergence of the block nonlinear Gauss Seidel method under convex constraints Operatons Research Letters 26 (2000) 127 136 www.elsever.com/locate/orms On the convergence of the bloc nonlnear Gauss Sedel method under convex constrants L. Grppo a, M. Scandrone b; a Dpartmento d Informatca

More information

IV. Performance Optimization

IV. Performance Optimization IV. Performance Optmzaton A. Steepest descent algorthm defnton how to set up bounds on learnng rate mnmzaton n a lne (varyng learnng rate) momentum learnng examples B. Newton s method defnton Gauss-Newton

More information

Generalized Linear Methods

Generalized Linear Methods Generalzed Lnear Methods 1 Introducton In the Ensemble Methods the general dea s that usng a combnaton of several weak learner one could make a better learner. More formally, assume that we have a set

More information

Lower Bounding Procedures for the Single Allocation Hub Location Problem

Lower Bounding Procedures for the Single Allocation Hub Location Problem Lower Boundng Procedures for the Snge Aocaton Hub Locaton Probem Borzou Rostam 1,2 Chrstoph Buchhem 1,4 Fautät für Mathemat, TU Dortmund, Germany J. Faban Meer 1,3 Uwe Causen 1 Insttute of Transport Logstcs,

More information

NUMERICAL DIFFERENTIATION

NUMERICAL DIFFERENTIATION NUMERICAL DIFFERENTIATION 1 Introducton Dfferentaton s a method to compute the rate at whch a dependent output y changes wth respect to the change n the ndependent nput x. Ths rate of change s called the

More information

Cyclic Codes BCH Codes

Cyclic Codes BCH Codes Cycc Codes BCH Codes Gaos Feds GF m A Gaos fed of m eements can be obtaned usng the symbos 0,, á, and the eements beng 0,, á, á, á 3 m,... so that fed F* s cosed under mutpcaton wth m eements. The operator

More information

Distributed Moving Horizon State Estimation of Nonlinear Systems. Jing Zhang

Distributed Moving Horizon State Estimation of Nonlinear Systems. Jing Zhang Dstrbuted Movng Horzon State Estmaton of Nonnear Systems by Jng Zhang A thess submtted n parta fufment of the requrements for the degree of Master of Scence n Chemca Engneerng Department of Chemca and

More information

More metrics on cartesian products

More metrics on cartesian products More metrcs on cartesan products If (X, d ) are metrc spaces for 1 n, then n Secton II4 of the lecture notes we defned three metrcs on X whose underlyng topologes are the product topology The purpose of

More information

Lecture Notes on Linear Regression

Lecture Notes on Linear Regression Lecture Notes on Lnear Regresson Feng L fl@sdueducn Shandong Unversty, Chna Lnear Regresson Problem In regresson problem, we am at predct a contnuous target value gven an nput feature vector We assume

More information

Linear Approximation with Regularization and Moving Least Squares

Linear Approximation with Regularization and Moving Least Squares Lnear Approxmaton wth Regularzaton and Movng Least Squares Igor Grešovn May 007 Revson 4.6 (Revson : March 004). 5 4 3 0.5 3 3.5 4 Contents: Lnear Fttng...4. Weghted Least Squares n Functon Approxmaton...

More information

LOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin

LOW BIAS INTEGRATED PATH ESTIMATORS. James M. Calvin Proceedngs of the 007 Wnter Smulaton Conference S G Henderson, B Bller, M-H Hseh, J Shortle, J D Tew, and R R Barton, eds LOW BIAS INTEGRATED PATH ESTIMATORS James M Calvn Department of Computer Scence

More information

Supplementary Material: Learning Structured Weight Uncertainty in Bayesian Neural Networks

Supplementary Material: Learning Structured Weight Uncertainty in Bayesian Neural Networks Shengyang Sun, Changyou Chen, Lawrence Carn Suppementary Matera: Learnng Structured Weght Uncertanty n Bayesan Neura Networks Shengyang Sun Changyou Chen Lawrence Carn Tsnghua Unversty Duke Unversty Duke

More information

Key words. corner singularities, energy-corrected finite element methods, optimal convergence rates, pollution effect, re-entrant corners

Key words. corner singularities, energy-corrected finite element methods, optimal convergence rates, pollution effect, re-entrant corners NESTED NEWTON STRATEGIES FOR ENERGY-CORRECTED FINITE ELEMENT METHODS U. RÜDE1, C. WALUGA 2, AND B. WOHLMUTH 2 Abstract. Energy-corrected fnte eement methods provde an attractve technque to dea wth eptc

More information

2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification

2E Pattern Recognition Solutions to Introduction to Pattern Recognition, Chapter 2: Bayesian pattern classification E395 - Pattern Recognton Solutons to Introducton to Pattern Recognton, Chapter : Bayesan pattern classfcaton Preface Ths document s a soluton manual for selected exercses from Introducton to Pattern Recognton

More information

Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur

Module 3 LOSSY IMAGE COMPRESSION SYSTEMS. Version 2 ECE IIT, Kharagpur Module 3 LOSSY IMAGE COMPRESSION SYSTEMS Verson ECE IIT, Kharagpur Lesson 6 Theory of Quantzaton Verson ECE IIT, Kharagpur Instructonal Objectves At the end of ths lesson, the students should be able to:

More information

Inexact Newton Methods for Inverse Eigenvalue Problems

Inexact Newton Methods for Inverse Eigenvalue Problems Inexact Newton Methods for Inverse Egenvalue Problems Zheng-jan Ba Abstract In ths paper, we survey some of the latest development n usng nexact Newton-lke methods for solvng nverse egenvalue problems.

More information

Neural network-based athletics performance prediction optimization model applied research

Neural network-based athletics performance prediction optimization model applied research Avaabe onne www.jocpr.com Journa of Chemca and Pharmaceutca Research, 04, 6(6):8-5 Research Artce ISSN : 0975-784 CODEN(USA) : JCPRC5 Neura networ-based athetcs performance predcton optmzaton mode apped

More information

Feature Selection: Part 1

Feature Selection: Part 1 CSE 546: Machne Learnng Lecture 5 Feature Selecton: Part 1 Instructor: Sham Kakade 1 Regresson n the hgh dmensonal settng How do we learn when the number of features d s greater than the sample sze n?

More information

MMA and GCMMA two methods for nonlinear optimization

MMA and GCMMA two methods for nonlinear optimization MMA and GCMMA two methods for nonlnear optmzaton Krster Svanberg Optmzaton and Systems Theory, KTH, Stockholm, Sweden. krlle@math.kth.se Ths note descrbes the algorthms used n the author s 2007 mplementatons

More information

Associative Memories

Associative Memories Assocatve Memores We consder now modes for unsupervsed earnng probems, caed auto-assocaton probems. Assocaton s the task of mappng patterns to patterns. In an assocatve memory the stmuus of an ncompete

More information

On the Power Function of the Likelihood Ratio Test for MANOVA

On the Power Function of the Likelihood Ratio Test for MANOVA Journa of Mutvarate Anayss 8, 416 41 (00) do:10.1006/jmva.001.036 On the Power Functon of the Lkehood Rato Test for MANOVA Dua Kumar Bhaumk Unversty of South Aabama and Unversty of Inos at Chcago and Sanat

More information

3. Stress-strain relationships of a composite layer

3. Stress-strain relationships of a composite layer OM PO I O U P U N I V I Y O F W N ompostes ourse 8-9 Unversty of wente ng. &ech... tress-stran reatonshps of a composte ayer - Laurent Warnet & emo Aerman.. tress-stran reatonshps of a composte ayer Introducton

More information

Asymptotics of the Solution of a Boundary Value. Problem for One-Characteristic Differential. Equation Degenerating into a Parabolic Equation

Asymptotics of the Solution of a Boundary Value. Problem for One-Characteristic Differential. Equation Degenerating into a Parabolic Equation Nonl. Analyss and Dfferental Equatons, ol., 4, no., 5 - HIKARI Ltd, www.m-har.com http://dx.do.org/.988/nade.4.456 Asymptotcs of the Soluton of a Boundary alue Problem for One-Characterstc Dfferental Equaton

More information

Chapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems

Chapter 5. Solution of System of Linear Equations. Module No. 6. Solution of Inconsistent and Ill Conditioned Systems Numercal Analyss by Dr. Anta Pal Assstant Professor Department of Mathematcs Natonal Insttute of Technology Durgapur Durgapur-713209 emal: anta.bue@gmal.com 1 . Chapter 5 Soluton of System of Lnear Equatons

More information

Vector Norms. Chapter 7 Iterative Techniques in Matrix Algebra. Cauchy-Bunyakovsky-Schwarz Inequality for Sums. Distances. Convergence.

Vector Norms. Chapter 7 Iterative Techniques in Matrix Algebra. Cauchy-Bunyakovsky-Schwarz Inequality for Sums. Distances. Convergence. Vector Norms Chapter 7 Iteratve Technques n Matrx Algebra Per-Olof Persson persson@berkeley.edu Department of Mathematcs Unversty of Calforna, Berkeley Math 128B Numercal Analyss Defnton A vector norm

More information

On a direct solver for linear least squares problems

On a direct solver for linear least squares problems ISSN 2066-6594 Ann. Acad. Rom. Sc. Ser. Math. Appl. Vol. 8, No. 2/2016 On a drect solver for lnear least squares problems Constantn Popa Abstract The Null Space (NS) algorthm s a drect solver for lnear

More information

Perron Vectors of an Irreducible Nonnegative Interval Matrix

Perron Vectors of an Irreducible Nonnegative Interval Matrix Perron Vectors of an Irreducble Nonnegatve Interval Matrx Jr Rohn August 4 2005 Abstract As s well known an rreducble nonnegatve matrx possesses a unquely determned Perron vector. As the man result of

More information

MAXIMUM NORM STABILITY OF DIFFERENCE SCHEMES FOR PARABOLIC EQUATIONS ON OVERSET NONMATCHING SPACE-TIME GRIDS

MAXIMUM NORM STABILITY OF DIFFERENCE SCHEMES FOR PARABOLIC EQUATIONS ON OVERSET NONMATCHING SPACE-TIME GRIDS MATHEMATICS OF COMPUTATION Voume 72 Number 242 Pages 619 656 S 0025-57180201462-X Artce eectroncay pubshed on November 4 2002 MAXIMUM NORM STABILITY OF DIFFERENCE SCHEMES FOR PARABOLIC EQUATIONS ON OVERSET

More information

Journal of Multivariate Analysis

Journal of Multivariate Analysis Journa of Mutvarate Anayss 3 (04) 74 96 Contents sts avaabe at ScenceDrect Journa of Mutvarate Anayss journa homepage: www.esever.com/ocate/jmva Hgh-dmensona sparse MANOVA T. Tony Ca a, Yn Xa b, a Department

More information

APPENDIX A Some Linear Algebra

APPENDIX A Some Linear Algebra APPENDIX A Some Lnear Algebra The collecton of m, n matrces A.1 Matrces a 1,1,..., a 1,n A = a m,1,..., a m,n wth real elements a,j s denoted by R m,n. If n = 1 then A s called a column vector. Smlarly,

More information

L-Edge Chromatic Number Of A Graph

L-Edge Chromatic Number Of A Graph IJISET - Internatona Journa of Innovatve Scence Engneerng & Technoogy Vo. 3 Issue 3 March 06. ISSN 348 7968 L-Edge Chromatc Number Of A Graph Dr.R.B.Gnana Joth Assocate Professor of Mathematcs V.V.Vannaperuma

More information

Some modelling aspects for the Matlab implementation of MMA

Some modelling aspects for the Matlab implementation of MMA Some modellng aspects for the Matlab mplementaton of MMA Krster Svanberg krlle@math.kth.se Optmzaton and Systems Theory Department of Mathematcs KTH, SE 10044 Stockholm September 2004 1. Consdered optmzaton

More information

Errors for Linear Systems

Errors for Linear Systems Errors for Lnear Systems When we solve a lnear system Ax b we often do not know A and b exactly, but have only approxmatons  and ˆb avalable. Then the best thng we can do s to solve ˆx ˆb exactly whch

More information

COXREG. Estimation (1)

COXREG. Estimation (1) COXREG Cox (972) frst suggested the modes n whch factors reated to fetme have a mutpcatve effect on the hazard functon. These modes are caed proportona hazards (PH) modes. Under the proportona hazards

More information

On the Multicriteria Integer Network Flow Problem

On the Multicriteria Integer Network Flow Problem BULGARIAN ACADEMY OF SCIENCES CYBERNETICS AND INFORMATION TECHNOLOGIES Volume 5, No 2 Sofa 2005 On the Multcrtera Integer Network Flow Problem Vassl Vasslev, Marana Nkolova, Maryana Vassleva Insttute of

More information

Week3, Chapter 4. Position and Displacement. Motion in Two Dimensions. Instantaneous Velocity. Average Velocity

Week3, Chapter 4. Position and Displacement. Motion in Two Dimensions. Instantaneous Velocity. Average Velocity Week3, Chapter 4 Moton n Two Dmensons Lecture Quz A partcle confned to moton along the x axs moves wth constant acceleraton from x =.0 m to x = 8.0 m durng a 1-s tme nterval. The velocty of the partcle

More information

n α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0

n α j x j = 0 j=1 has a nontrivial solution. Here A is the n k matrix whose jth column is the vector for all t j=0 MODULE 2 Topcs: Lnear ndependence, bass and dmenson We have seen that f n a set of vectors one vector s a lnear combnaton of the remanng vectors n the set then the span of the set s unchanged f that vector

More information

Problem Set 9 Solutions

Problem Set 9 Solutions Desgn and Analyss of Algorthms May 4, 2015 Massachusetts Insttute of Technology 6.046J/18.410J Profs. Erk Demane, Srn Devadas, and Nancy Lynch Problem Set 9 Solutons Problem Set 9 Solutons Ths problem

More information

A MIN-MAX REGRET ROBUST OPTIMIZATION APPROACH FOR LARGE SCALE FULL FACTORIAL SCENARIO DESIGN OF DATA UNCERTAINTY

A MIN-MAX REGRET ROBUST OPTIMIZATION APPROACH FOR LARGE SCALE FULL FACTORIAL SCENARIO DESIGN OF DATA UNCERTAINTY A MIN-MAX REGRET ROBST OPTIMIZATION APPROACH FOR ARGE SCAE F FACTORIA SCENARIO DESIGN OF DATA NCERTAINTY Travat Assavapokee Department of Industra Engneerng, nversty of Houston, Houston, Texas 7704-4008,

More information

[WAVES] 1. Waves and wave forces. Definition of waves

[WAVES] 1. Waves and wave forces. Definition of waves 1. Waves and forces Defnton of s In the smuatons on ong-crested s are consdered. The drecton of these s (μ) s defned as sketched beow n the goba co-ordnate sstem: North West East South The eevaton can

More information

The line method combined with spectral chebyshev for space-time fractional diffusion equation

The line method combined with spectral chebyshev for space-time fractional diffusion equation Apped and Computatona Mathematcs 014; 3(6): 330-336 Pubshed onne December 31, 014 (http://www.scencepubshnggroup.com/j/acm) do: 10.1164/j.acm.0140306.17 ISS: 3-5605 (Prnt); ISS: 3-5613 (Onne) The ne method

More information

A Class of Distributed Optimization Methods with Event-Triggered Communication

A Class of Distributed Optimization Methods with Event-Triggered Communication A Cass of Dstrbuted Optmzaton Methods wth Event-Trggered Communcaton Martn C. Mene Mchae Ubrch Sebastan Abrecht the date of recept and acceptance shoud be nserted ater Abstract We present a cass of methods

More information

College of Computer & Information Science Fall 2009 Northeastern University 20 October 2009

College of Computer & Information Science Fall 2009 Northeastern University 20 October 2009 College of Computer & Informaton Scence Fall 2009 Northeastern Unversty 20 October 2009 CS7880: Algorthmc Power Tools Scrbe: Jan Wen and Laura Poplawsk Lecture Outlne: Prmal-dual schema Network Desgn:

More information

Numerical Heat and Mass Transfer

Numerical Heat and Mass Transfer Master degree n Mechancal Engneerng Numercal Heat and Mass Transfer 06-Fnte-Dfference Method (One-dmensonal, steady state heat conducton) Fausto Arpno f.arpno@uncas.t Introducton Why we use models and

More information

3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X

3.1 Expectation of Functions of Several Random Variables. )' be a k-dimensional discrete or continuous random vector, with joint PMF p (, E X E X1 E X Statstcs 1: Probablty Theory II 37 3 EPECTATION OF SEVERAL RANDOM VARIABLES As n Probablty Theory I, the nterest n most stuatons les not on the actual dstrbuton of a random vector, but rather on a number

More information

REAL-TIME IMPACT FORCE IDENTIFICATION OF CFRP LAMINATED PLATES USING SOUND WAVES

REAL-TIME IMPACT FORCE IDENTIFICATION OF CFRP LAMINATED PLATES USING SOUND WAVES 8 TH INTERNATIONAL CONERENCE ON COMPOSITE MATERIALS REAL-TIME IMPACT ORCE IDENTIICATION O CRP LAMINATED PLATES USING SOUND WAVES S. Atobe *, H. Kobayash, N. Hu 3 and H. ukunaga Department of Aerospace

More information

VQ widely used in coding speech, image, and video

VQ widely used in coding speech, image, and video at Scalar quantzers are specal cases of vector quantzers (VQ): they are constraned to look at one sample at a tme (memoryless) VQ does not have such constrant better RD perfomance expected Source codng

More information

Monica Purcaru and Nicoleta Aldea. Abstract

Monica Purcaru and Nicoleta Aldea. Abstract FILOMAT (Nš) 16 (22), 7 17 GENERAL CONFORMAL ALMOST SYMPLECTIC N-LINEAR CONNECTIONS IN THE BUNDLE OF ACCELERATIONS Monca Purcaru and Ncoeta Adea Abstract The am of ths paper 1 s to fnd the transformaton

More information

Foundations of Arithmetic

Foundations of Arithmetic Foundatons of Arthmetc Notaton We shall denote the sum and product of numbers n the usual notaton as a 2 + a 2 + a 3 + + a = a, a 1 a 2 a 3 a = a The notaton a b means a dvdes b,.e. ac = b where c s an

More information

A parametric Linear Programming Model Describing Bandwidth Sharing Policies for ABR Traffic

A parametric Linear Programming Model Describing Bandwidth Sharing Policies for ABR Traffic parametrc Lnear Programmng Mode Descrbng Bandwdth Sharng Poces for BR Traffc I. Moschoos, M. Logothets and G. Kokknaks Wre ommuncatons Laboratory, Dept. of Eectrca & omputer Engneerng, Unversty of Patras,

More information

LECTURE 21 Mohr s Method for Calculation of General Displacements. 1 The Reciprocal Theorem

LECTURE 21 Mohr s Method for Calculation of General Displacements. 1 The Reciprocal Theorem V. DEMENKO MECHANICS OF MATERIALS 05 LECTURE Mohr s Method for Cacuaton of Genera Dspacements The Recproca Theorem The recproca theorem s one of the genera theorems of strength of materas. It foows drect

More information

Appendix for An Efficient Ascending-Bid Auction for Multiple Objects: Comment For Online Publication

Appendix for An Efficient Ascending-Bid Auction for Multiple Objects: Comment For Online Publication Appendx for An Effcent Ascendng-Bd Aucton for Mutpe Objects: Comment For Onne Pubcaton Norak Okamoto The foowng counterexampe shows that sncere bddng by a bdders s not aways an ex post perfect equbrum

More information

G : Statistical Mechanics

G : Statistical Mechanics G25.2651: Statstca Mechancs Notes for Lecture 11 I. PRINCIPLES OF QUANTUM STATISTICAL MECHANICS The probem of quantum statstca mechancs s the quantum mechanca treatment of an N-partce system. Suppose the

More information

Singular Value Decomposition: Theory and Applications

Singular Value Decomposition: Theory and Applications Sngular Value Decomposton: Theory and Applcatons Danel Khashab Sprng 2015 Last Update: March 2, 2015 1 Introducton A = UDV where columns of U and V are orthonormal and matrx D s dagonal wth postve real

More information

Lower bounds for the Crossing Number of the Cartesian Product of a Vertex-transitive Graph with a Cycle

Lower bounds for the Crossing Number of the Cartesian Product of a Vertex-transitive Graph with a Cycle Lower bounds for the Crossng Number of the Cartesan Product of a Vertex-transtve Graph wth a Cyce Junho Won MIT-PRIMES December 4, 013 Abstract. The mnmum number of crossngs for a drawngs of a gven graph

More information

n-step cycle inequalities: facets for continuous n-mixing set and strong cuts for multi-module capacitated lot-sizing problem

n-step cycle inequalities: facets for continuous n-mixing set and strong cuts for multi-module capacitated lot-sizing problem n-step cyce nequates: facets for contnuous n-mxng set and strong cuts for mut-modue capactated ot-szng probem Mansh Bansa and Kavash Kanfar Department of Industra and Systems Engneerng, Texas A&M Unversty,

More information

A DIMENSION-REDUCTION METHOD FOR STOCHASTIC ANALYSIS SECOND-MOMENT ANALYSIS

A DIMENSION-REDUCTION METHOD FOR STOCHASTIC ANALYSIS SECOND-MOMENT ANALYSIS A DIMESIO-REDUCTIO METHOD FOR STOCHASTIC AALYSIS SECOD-MOMET AALYSIS S. Rahman Department of Mechanca Engneerng and Center for Computer-Aded Desgn The Unversty of Iowa Iowa Cty, IA 52245 June 2003 OUTLIE

More information

Xin Li Department of Information Systems, College of Business, City University of Hong Kong, Hong Kong, CHINA

Xin Li Department of Information Systems, College of Business, City University of Hong Kong, Hong Kong, CHINA RESEARCH ARTICLE MOELING FIXE OS BETTING FOR FUTURE EVENT PREICTION Weyun Chen eartment of Educatona Informaton Technoogy, Facuty of Educaton, East Chna Norma Unversty, Shangha, CHINA {weyun.chen@qq.com}

More information

Subgradient Methods and Consensus Algorithms for Solving Convex Optimization Problems

Subgradient Methods and Consensus Algorithms for Solving Convex Optimization Problems Proceedngs of the 47th IEEE Conference on Decson and Contro Cancun, Mexco, Dec. 9-11, 2008 Subgradent Methods and Consensus Agorthms for Sovng Convex Optmzaton Probems Björn Johansson, Tamás Kevczy, Mae

More information

Quantum Runge-Lenz Vector and the Hydrogen Atom, the hidden SO(4) symmetry

Quantum Runge-Lenz Vector and the Hydrogen Atom, the hidden SO(4) symmetry Quantum Runge-Lenz ector and the Hydrogen Atom, the hdden SO(4) symmetry Pasca Szrftgser and Edgardo S. Cheb-Terrab () Laboratore PhLAM, UMR CNRS 85, Unversté Le, F-59655, France () Mapesoft Let's consder

More information

The Minimum Universal Cost Flow in an Infeasible Flow Network

The Minimum Universal Cost Flow in an Infeasible Flow Network Journal of Scences, Islamc Republc of Iran 17(2): 175-180 (2006) Unversty of Tehran, ISSN 1016-1104 http://jscencesutacr The Mnmum Unversal Cost Flow n an Infeasble Flow Network H Saleh Fathabad * M Bagheran

More information

Using T.O.M to Estimate Parameter of distributions that have not Single Exponential Family

Using T.O.M to Estimate Parameter of distributions that have not Single Exponential Family IOSR Journal of Mathematcs IOSR-JM) ISSN: 2278-5728. Volume 3, Issue 3 Sep-Oct. 202), PP 44-48 www.osrjournals.org Usng T.O.M to Estmate Parameter of dstrbutons that have not Sngle Exponental Famly Jubran

More information

Predicting Model of Traffic Volume Based on Grey-Markov

Predicting Model of Traffic Volume Based on Grey-Markov Vo. No. Modern Apped Scence Predctng Mode of Traffc Voume Based on Grey-Marov Ynpeng Zhang Zhengzhou Muncpa Engneerng Desgn & Research Insttute Zhengzhou 5005 Chna Abstract Grey-marov forecastng mode of

More information

A new construction of 3-separable matrices via an improved decoding of Macula s construction

A new construction of 3-separable matrices via an improved decoding of Macula s construction Dscrete Optmzaton 5 008 700 704 Contents lsts avalable at ScenceDrect Dscrete Optmzaton journal homepage: wwwelsevercom/locate/dsopt A new constructon of 3-separable matrces va an mproved decodng of Macula

More information

Note 2. Ling fong Li. 1 Klein Gordon Equation Probablity interpretation Solutions to Klein-Gordon Equation... 2

Note 2. Ling fong Li. 1 Klein Gordon Equation Probablity interpretation Solutions to Klein-Gordon Equation... 2 Note 2 Lng fong L Contents Ken Gordon Equaton. Probabty nterpretaton......................................2 Soutons to Ken-Gordon Equaton............................... 2 2 Drac Equaton 3 2. Probabty nterpretaton.....................................

More information

Solutions to exam in SF1811 Optimization, Jan 14, 2015

Solutions to exam in SF1811 Optimization, Jan 14, 2015 Solutons to exam n SF8 Optmzaton, Jan 4, 25 3 3 O------O -4 \ / \ / The network: \/ where all lnks go from left to rght. /\ / \ / \ 6 O------O -5 2 4.(a) Let x = ( x 3, x 4, x 23, x 24 ) T, where the varable

More information

CIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M

CIS526: Machine Learning Lecture 3 (Sept 16, 2003) Linear Regression. Preparation help: Xiaoying Huang. x 1 θ 1 output... θ M x M CIS56: achne Learnng Lecture 3 (Sept 6, 003) Preparaton help: Xaoyng Huang Lnear Regresson Lnear regresson can be represented by a functonal form: f(; θ) = θ 0 0 +θ + + θ = θ = 0 ote: 0 s a dummy attrbute

More information

2.3 Nilpotent endomorphisms

2.3 Nilpotent endomorphisms s a block dagonal matrx, wth A Mat dm U (C) In fact, we can assume that B = B 1 B k, wth B an ordered bass of U, and that A = [f U ] B, where f U : U U s the restrcton of f to U 40 23 Nlpotent endomorphsms

More information

Genericity of Critical Types

Genericity of Critical Types Genercty of Crtcal Types Y-Chun Chen Alfredo D Tllo Eduardo Fangold Syang Xong September 2008 Abstract Ely and Pesk 2008 offers an nsghtful characterzaton of crtcal types: a type s crtcal f and only f

More information

Achieving Optimal Throughput Utility and Low Delay with CSMA-like Algorithms: A Virtual Multi-Channel Approach

Achieving Optimal Throughput Utility and Low Delay with CSMA-like Algorithms: A Virtual Multi-Channel Approach Achevng Optma Throughput Utty and Low Deay wth SMA-ke Agorthms: A Vrtua Mut-hanne Approach Po-Ka Huang, Student Member, IEEE, and Xaojun Ln, Senor Member, IEEE Abstract SMA agorthms have recenty receved

More information

Report on Image warping

Report on Image warping Report on Image warpng Xuan Ne, Dec. 20, 2004 Ths document summarzed the algorthms of our mage warpng soluton for further study, and there s a detaled descrpton about the mplementaton of these algorthms.

More information

Which Separator? Spring 1

Which Separator? Spring 1 Whch Separator? 6.034 - Sprng 1 Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng Whch Separator? Mamze the margn to closest ponts 6.034 - Sprng 3 Margn of a pont " # y (w $ + b) proportonal

More information

Physics 5153 Classical Mechanics. D Alembert s Principle and The Lagrangian-1

Physics 5153 Classical Mechanics. D Alembert s Principle and The Lagrangian-1 P. Guterrez Physcs 5153 Classcal Mechancs D Alembert s Prncple and The Lagrangan 1 Introducton The prncple of vrtual work provdes a method of solvng problems of statc equlbrum wthout havng to consder the

More information

ON THE EXTENDED HAAGERUP TENSOR PRODUCT IN OPERATOR SPACES. 1. Introduction

ON THE EXTENDED HAAGERUP TENSOR PRODUCT IN OPERATOR SPACES. 1. Introduction ON THE EXTENDED HAAGERUP TENSOR PRODUCT IN OPERATOR SPACES TAKASHI ITOH AND MASARU NAGISA Abstract We descrbe the Haagerup tensor product l h l and the extended Haagerup tensor product l eh l n terms of

More information

Assortment Optimization under MNL

Assortment Optimization under MNL Assortment Optmzaton under MNL Haotan Song Aprl 30, 2017 1 Introducton The assortment optmzaton problem ams to fnd the revenue-maxmzng assortment of products to offer when the prces of products are fxed.

More information

6. Stochastic processes (2)

6. Stochastic processes (2) Contents Markov processes Brth-death processes Lect6.ppt S-38.45 - Introducton to Teletraffc Theory Sprng 5 Markov process Consder a contnuous-tme and dscrete-state stochastc process X(t) wth state space

More information

Affine transformations and convexity

Affine transformations and convexity Affne transformatons and convexty The purpose of ths document s to prove some basc propertes of affne transformatons nvolvng convex sets. Here are a few onlne references for background nformaton: http://math.ucr.edu/

More information

PHYS 705: Classical Mechanics. Calculus of Variations II

PHYS 705: Classical Mechanics. Calculus of Variations II 1 PHYS 705: Classcal Mechancs Calculus of Varatons II 2 Calculus of Varatons: Generalzaton (no constrant yet) Suppose now that F depends on several dependent varables : We need to fnd such that has a statonary

More information

6. Stochastic processes (2)

6. Stochastic processes (2) 6. Stochastc processes () Lect6.ppt S-38.45 - Introducton to Teletraffc Theory Sprng 5 6. Stochastc processes () Contents Markov processes Brth-death processes 6. Stochastc processes () Markov process

More information

Société de Calcul Mathématique SA

Société de Calcul Mathématique SA Socété de Calcul Mathématque SA Outls d'ade à la décson Tools for decson help Probablstc Studes: Normalzng the Hstograms Bernard Beauzamy December, 202 I. General constructon of the hstogram Any probablstc

More information

Perfect Competition and the Nash Bargaining Solution

Perfect Competition and the Nash Bargaining Solution Perfect Competton and the Nash Barganng Soluton Renhard John Department of Economcs Unversty of Bonn Adenauerallee 24-42 53113 Bonn, Germany emal: rohn@un-bonn.de May 2005 Abstract For a lnear exchange

More information

Some Comments on Accelerating Convergence of Iterative Sequences Using Direct Inversion of the Iterative Subspace (DIIS)

Some Comments on Accelerating Convergence of Iterative Sequences Using Direct Inversion of the Iterative Subspace (DIIS) Some Comments on Acceleratng Convergence of Iteratve Sequences Usng Drect Inverson of the Iteratve Subspace (DIIS) C. Davd Sherrll School of Chemstry and Bochemstry Georga Insttute of Technology May 1998

More information

Interference Alignment and Degrees of Freedom Region of Cellular Sigma Channel

Interference Alignment and Degrees of Freedom Region of Cellular Sigma Channel 2011 IEEE Internatona Symposum on Informaton Theory Proceedngs Interference Agnment and Degrees of Freedom Regon of Ceuar Sgma Channe Huaru Yn 1 Le Ke 2 Zhengdao Wang 2 1 WINLAB Dept of EEIS Unv. of Sc.

More information

Example: Suppose we want to build a classifier that recognizes WebPages of graduate students.

Example: Suppose we want to build a classifier that recognizes WebPages of graduate students. Exampe: Suppose we want to bud a cassfer that recognzes WebPages of graduate students. How can we fnd tranng data? We can browse the web and coect a sampe of WebPages of graduate students of varous unverstes.

More information

Approximate merging of a pair of BeÂzier curves

Approximate merging of a pair of BeÂzier curves COMPUTER-AIDED DESIGN Computer-Aded Desgn 33 (1) 15±136 www.esever.com/ocate/cad Approxmate mergng of a par of BeÂzer curves Sh-Mn Hu a,b, *, Rou-Feng Tong c, Tao Ju a,b, Ja-Guang Sun a,b a Natona CAD

More information

A generalization of Picard-Lindelof theorem/ the method of characteristics to systems of PDE

A generalization of Picard-Lindelof theorem/ the method of characteristics to systems of PDE A generazaton of Pcard-Lndeof theorem/ the method of characterstcs to systems of PDE Erfan Shachan b,a,1 a Department of Physcs, Unversty of Toronto, 60 St. George St., Toronto ON M5S 1A7, Canada b Department

More information

Maximizing the number of nonnegative subsets

Maximizing the number of nonnegative subsets Maxmzng the number of nonnegatve subsets Noga Alon Hao Huang December 1, 213 Abstract Gven a set of n real numbers, f the sum of elements of every subset of sze larger than k s negatve, what s the maxmum

More information

Application of B-Spline to Numerical Solution of a System of Singularly Perturbed Problems

Application of B-Spline to Numerical Solution of a System of Singularly Perturbed Problems Mathematca Aeterna, Vol. 1, 011, no. 06, 405 415 Applcaton of B-Splne to Numercal Soluton of a System of Sngularly Perturbed Problems Yogesh Gupta Department of Mathematcs Unted College of Engneerng &

More information

Estimating the Fundamental Matrix by Transforming Image Points in Projective Space 1

Estimating the Fundamental Matrix by Transforming Image Points in Projective Space 1 Estmatng the Fundamental Matrx by Transformng Image Ponts n Projectve Space 1 Zhengyou Zhang and Charles Loop Mcrosoft Research, One Mcrosoft Way, Redmond, WA 98052, USA E-mal: fzhang,cloopg@mcrosoft.com

More information

LINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity

LINEAR REGRESSION ANALYSIS. MODULE IX Lecture Multicollinearity LINEAR REGRESSION ANALYSIS MODULE IX Lecture - 30 Multcollnearty Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur 2 Remedes for multcollnearty Varous technques have

More information

APPROXIMATE PRICES OF BASKET AND ASIAN OPTIONS DUPONT OLIVIER. Premia 14

APPROXIMATE PRICES OF BASKET AND ASIAN OPTIONS DUPONT OLIVIER. Premia 14 APPROXIMAE PRICES OF BASKE AND ASIAN OPIONS DUPON OLIVIER Prema 14 Contents Introducton 1 1. Framewor 1 1.1. Baset optons 1.. Asan optons. Computng the prce 3. Lower bound 3.1. Closed formula for the prce

More information

princeton univ. F 17 cos 521: Advanced Algorithm Design Lecture 7: LP Duality Lecturer: Matt Weinberg

princeton univ. F 17 cos 521: Advanced Algorithm Design Lecture 7: LP Duality Lecturer: Matt Weinberg prnceton unv. F 17 cos 521: Advanced Algorthm Desgn Lecture 7: LP Dualty Lecturer: Matt Wenberg Scrbe: LP Dualty s an extremely useful tool for analyzng structural propertes of lnear programs. Whle there

More information

Linear, affine, and convex sets and hulls In the sequel, unless otherwise specified, X will denote a real vector space.

Linear, affine, and convex sets and hulls In the sequel, unless otherwise specified, X will denote a real vector space. Lnear, affne, and convex sets and hulls In the sequel, unless otherwse specfed, X wll denote a real vector space. Lnes and segments. Gven two ponts x, y X, we defne xy = {x + t(y x) : t R} = {(1 t)x +

More information