An Accelerated Proximal Coordinate Gradient Method
|
|
- Hilary Thomasina Goodman
- 5 years ago
- Views:
Transcription
1 A Accelerated Proxmal Coordate Gradet Method Qhag L Uversty of Iowa Iowa Cty IA USA qhag-l@uowaedu Zhaosog Lu Smo Fraser Uversty Buraby BC Caada zhaosog@sfuca L Xao Mcrosoft Research Redmod WA USA lxao@mcrosoftcom Abstract We develop a accelerated radomzed proxmal coordate gradet APCG method for solvg a broad class of composte covex optmzato problems I partcular our method acheves faster lear covergece rates for mmzg strogly covex fuctos tha exstg radomzed proxmal coordate gradet methods We show how to apply the APCG method to solve the dual of the regularzed emprcal rsk mmzato ERM problem ad devse effcet mplemetatos that avod full-dmesoal vector operatos For ll-codtoed ERM problems our method obtas mproved covergece rates tha the state-ofthe-art stochastc dual coordate ascet SDCA method 1 Itroducto Coordate descet methods have receved extesve atteto recet years due to ther potetal for solvg large-scale optmzato problems arsg from mache learg ad other applcatos I ths paper we develop a accelerated proxmal coordate gradet APCG method for solvg covex optmzato problems wth the followg form: mmze x R N Fx def = fx+ψx } 1 where f s dfferetable o domψ ad Ψ has a block separable structure More specfcally Ψx = Ψ x where each x deotes a sub-vector of x wth cardalty N ad each Ψ : R N R + } s a closed covex fucto We assume the collecto x : = 1} form a partto of the compoets of x R N I addto to the capablty of modelg osmooth regularzato terms such as Ψx = λ x 1 ths model also cludes optmzato problems wth block separable costrats More precsely each block costrat x C where C s a closed covex set ca be modeled by a dcator fucto defed as Ψ x = 0 fx C ad otherwse At each terato coordate descet methods choose oe block of coordates x to suffcetly reduce the objectve value whle keepg other blocks fxed Oe commo ad smple approach for choosg such a block s the cyclc scheme The global ad local covergece propertes of the cyclc coordate descet method have bee studed for example [ ] Recetly radomzed strateges for choosg the block to update became more popular I addto to ts theoretcal beefts [ ] umerous expermets have demostrated that radomzed coordate descet methods are very powerful for solvg large-scale mache learg problems [ ] Ispred by the success of accelerated full gradet methods eg [1 1 ] several recet work appled Nesterov s accelerato schemes to speed up radomzed coordate descet methods I partcular Nesterov [13] developed a accelerated radomzed coordate gradet method for mmzg ucostraed smooth covex fuctos whch correspods to the case of Ψx 0 1 1
2 Lu ad Xao [10] gave a sharper covergece aalyss of Nesterov s method ad Lee ad Sdford [8] developed extesos wth weghted radom samplg schemes More recetly Fercoq ad Rchtárk [4] proposed a APPROX Accelerated Parallel ad PROXmal coordate descet method for solvg the more geeral problem 1 ad obtaed accelerated sublear covergece rates but ther method caot explot the strog covexty to obta accelerated lear rates I ths paper we develop a geeral APCG method that acheves accelerated lear covergece rates whe the objectve fucto s strogly covex Wthout the strog covexty assumpto our method recovers the APPROX method [4] Moreover we show how to apply the APCG method to solve the dual of the regularzed emprcal rsk mmzato ERM problem ad devse effcet mplemetatos that avod full-dmesoal vector operatos For ll-codtoed ERM problems our method obtas faster covergece rates tha the state-of-the-art stochastc dual coordate ascet SDCA method [19] ad the mproved terato complexty matches the accelerated SDCA method [0] We preset umercal expermets to llustrate the advatage of our method 11 Notatos ad assumptos For ay partto of x R N to x R N : = 1} there s a N N permutato matrxu parttoed as U = [U 1 U ] where U R N N such that x = U x ad x = U T x = 1 For ay x R N the partal gradet of f wth respect tox s defed as fx = U T fx = 1 We assocate each subspace R N for = 1 wth the stadard Eucldea orm deoted by We make the followg assumptos whch are stadard the lterature o coordate descet methods eg [13 14] Assumpto 1 The gradet of fuctof s block-wse Lpschtz cotuous wth costatsl e fx+u h fx L h h R N = 1 x R N For coveece we defe the followg orm the whole space R N : 1/ x L = L x x R N 3 Assumpto There exsts µ 0 such that for ally R N ad x domψ fy fx+ fxy x + µ y x L The covexty parameter of f wth respect to the orm L s the largest µ such that the above equalty holds Every covex fucto satsfes Assumpto wthµ = 0 Ifµ > 0 the fuctof s called strogly covex We ote that a mmedate cosequece of Assumpto 1 s fx+u h fx+ fxh + L h h R N = 1 x R N 4 Ths together wth Assumpto mples µ 1 The APCG method I ths secto we descrbe the geeral APCG method ad several varats that are more sutable for mplemetato uder dfferet assumptos These algorthms exted Nesterov s accelerated gradet methods [1 Secto ] to the composte ad coordate descet settg We frst expla the otatos used our algorthms The algorthms proceed teratos wth k beg the terato couter Lower case letters x y z represet vectors the full space R N ad x k y k ad z k are ther values at the kth terato Each block coordate s dcated wth a subscrpt for examplex k represets the value of theth block of the vectorx k The Greek letters α β γ are scalars ad α k β k ad γ k represet ther values at teratok
3 Algorthm 1: the APCG method Iput: x 0 domψ ad covexty parameter µ 0 Italze: set z 0 = x 0 ad choose 0 < γ 0 [µ1] Iterate: repeat fork = 01 1 Compute α k 0 1 ] from the equato ad set Compute y k as y k = α k = 1 α k γ k +α k µ 5 γ k+1 = 1 α k γ k +α k µ β k = α kµ γ k α k γ k z k +γ k+1 x k 7 α k γ k +γ k+1 3 Choose k 1} uformly at radom ad compute z k+1 αk = argm x 1 β k z k β k y k } x R + L k fy k x k +Ψ k x k N 4 Set x k+1 = y k +α k z k+1 z k + µ zk y k 8 The geeral APCG method s gve as Algorthm 1 At each terato k t chooses a radom coordate k 1} ad geeratesy k x k+1 adz k+1 Oe ca observe thatx k+1 ad z k+1 deped o the realzato of the radom varable ξ k = 0 1 k } whle y k s depedet of k ad oly depeds o ξ k 1 To better uderstad ths method we make the followg observatos For coveece we defe z k+1 αk = argm x 1 βk z k β k y k } x R L + fyk x y k +Ψx 9 N whch s a full-dmesoal update verso of Step 3 Oe ca observe that z k+1 s updated as k+1 z f = k z k+1 = 1 β k z k Notce that from ad 8 we have whch together wth 10 yelds x k+1 = +β k y k f k x k+1 = y k +α k z k+1 1 β k z k β k y k y k +α k z k+1 z k + µ z k y k f = k y k f k That s Step 4 we oly eed to update the block coordates x k+1 k ad set the rest to bey k We ow state a theorem cocerg the expected rate of covergece of the APCG method whose proof ca be foud the full report [9] Theorem 1 Suppose Assumptos 1 ad hold Let F be the optmal value of problem 1 ad x k } be the sequece geerated by the APCG method The for ay k 0 there holds: µ k E ξk 1 [Fx k ] F m 1 +k γ 0 } Fx 0 F + γ 0 R 0 where def R 0 = m x x X x0 L 1 ad X s the set of optmal solutos of problem 1 3
4 Our result Theorem 1 mproves upo the covergece rates of the proxmal coordate gradet methods [14 10] whch have covergece rates o the order of 1 } µ k O m +k For = 1 our result matches exactly that of the accelerated full gradet method [1 Secto ] 1 Two specal cases Here we gve two smplfed versos of the APCG method for the specal cases of µ = 0 ad µ > 0 respectvely Algorthm shows the smplfed verso for µ = 0 whch ca be appled to problems wthout strog covexty or f the covexty parameter µ s ukow Iput: x 0 domψ Algorthm : APCG wthµ = 0 Italze: set z 0 = x 0 ad choose α ] Iterate: repeat fork = 01 1 Compute y k = 1 α k x k +α k z k Choose k 1} uformly at radom ad compute = argm x R N z k+1 k ad setz k+1 αk L k = z k for all k 3 Set x k+1 = y k +α k z k+1 z k α 4 Compute α k+1 = 1 4 k +4α k α k x z k k + k fy k x y k } k +Ψ k x Accordg to Theorem 1 Algorthm has a accelerated sublear covergece rate that s E ξk 1 [Fx k ] F Fx 0 F + 1 +kα 0 R 0 Wth the choce of α 0 = 1/ Algorthm reduces to the APPROX method [4] wth sgle block update at each terato eτ = 1 ther Algorthm 1 For the strogly covex case wth µ > 0 we ca talze Algorthm 1 wth the parameter γ 0 = µ whch mples γ k = µ ad α k = β k = µ/ for all k 0 Ths results Algorthm 3 Algorthm 3: APCG wthγ 0 = µ > 0 Iput: x 0 domψ ad covexty parameter µ > 0 Italze: set z 0 = x 0 ad ad α = µ Iterate: repeat fork = 01 1 Compute y k = xk +αz k 1+α Choose k 1} uformly at radom ad compute z k+1 α = argm x 1 αz k αy k } + L k fy k x k y k k +Ψ k x k x R N 3 Set x k+1 = y k +αz k+1 z k +α z k y k As a drect corollary of Theorem 1 Algorthm 3 ejoys a accelerated lear covergece rate: µ k E ξk 1 [Fx k ] F 1 Fx 0 F + µ R 0 To the best of our kowledge ths s the frst tme such a accelerated rate s obtaed for solvg the geeral problem 1 wth strog covexty usg coordate descet type of methods 4
5 Effcet mplemetato The APCG methods we preseted so far all eed to perform full-dmesoal vector operatos at each terato For example y k s updated as a covex combato of x k ad z k ad ths ca be very costly sce geeral they ca be dese vectors Moreover for the strogly covex case Algorthms 1 ad 3 all blocks of z k+1 eed to be updated at each terato although oly the k -th block eeds to compute the partal gradet ad perform a proxmal mappg These full-dmesoal vector updates cost ON operatos per terato ad may cause the overall computatoal cost of APCG to be eve hgher tha the full gradet methods see dscussos [13] I order to avod full-dmesoal vector operatos Lee ad Sdford [8] proposed a chage of varables scheme for accelerated coordated descet methods for ucostraed smooth mmzato Fercoq ad Rchtárk [4] devsed a smlar scheme for effcet mplemetato the µ = 0 case for composte mmzato Here we show that such a scheme ca also be developed for the case of µ > 0 the composte optmzato settg For smplcty we oly preset a equvalet mplemetato of the smplfed APCG method descrbed Algorthm 3 Algorthm 4: Effcet mplemetato of APCG wthγ 0 = µ > 0 Iput: x 0 domψ ad covexty parameter µ > 0 Italze: set α = µ 1 α ad ρ = 1+α ad talzeu0 = 0 ad v 0 = x 0 Iterate: repeat fork = 01 1 Choose k 1} uformly at radom ad compute } k αlk k = argm + k fρ k+1 u k +v k +Ψ k ρ k+1 u k k +v k k + R N k Let u k+1 = u k ad v k+1 = v k ad update u k+1 k = u k k 1 α ρ k+1 k k Output: x k+1 = ρ k+1 u k+1 +v k+1 v k+1 k = v k k + 1+α k k 13 The followg Proposto s proved the full report [9] Proposto 1 The terates of Algorthm 3 ad Algorthm 4 satsfy the followg relatoshps: x k = ρ k u k +v k y k = ρ k+1 u k +v k z k = ρ k u k +v k 14 We ote that Algorthm 4 oly a sgle block coordate of the vectorsu k adv k are updated at each terato whch coston However computg the partal gradet k fρ k+1 u k +v k may stll coston geeral I the ext secto we show how to further explot structure may ERM problems to completely avod full-dmesoal vector operatos 3 Applcato to regularzed emprcal rsk mmzato ERM LetA 1 A be vectors R d φ 1 φ be a sequece of covex fuctos defed or adg be a covex fucto o R d Regularzed ERM ams to solve the followg problem: mmze w R d Pw wth Pw = 1 φ A T w+λgw whereλ > 0 s a regularzato parameter For example gve a labelb ±1} for each vectora for = 1 we obta the lear SVM problem by settgφ z = max01 b z} adgw = 1/ w Regularzed logstc regresso s obtaed by settgφ z = log1+exp b z Ths formulato also cludes regresso problems For example rdge regresso s obtaed by settg 1/φ z = z b ad gw = 1/ w ad we get Lasso fgw = w 1 5
6 Let φ be the covex cojugate of φ that s φ u = max z Rzu φ z The dual of the regularzed ERM problem s see eg [19] maxmze Dx wth Dx = 1 1 φ x R x λg λ Ax where A = [A 1 A ] Ths s equvalet to mmze Fx def = Dx that s mmze x R Fx def = 1 φ x +λg 1 λ Ax The structure of Fx above matches the formulato 1 ad wth fx = λg 1 λ Ax ad Ψ x = 1 φ x ad we ca apply the APCG method to mmze Fx I order to explot the fast lear covergece rate we make the followg assumpto Assumpto 3 Each fuctoφ s1/γ smooth ad the fuctog has ut covexty parameter 1 Here we slghtly abuse the otato by overloadg γ whch also appeared Algorthm 1 But ths secto t solely represets the verse smoothess parameter of φ Assumpto 3 mples that each φ has strog covexty parameter γ wth respect to the local Eucldea orm ad g s dfferetable ad g has Lpschtz costat 1 I the followg we splt the fucto Fx = fx+ψx by relocatg the strog covexty term as follows: 1 fx = λg λ Ax + γ x Ψx = 1 φ x γ x 15 As a result the fucto f s strogly covex ad each Ψ s stll covex Now we ca apply the APCG method to mmzefx = Dx ad obta the followg guaratee Theorem Suppose Assumpto 3 holds ad A R for all = 1 I order to obta a expected dual optmalty gap E[D Dx k ] ǫ by usg the APCG method t suffces to have k + R logc/ǫ 16 where D = max x R Dx ad the costat C = D Dx 0 +γ/ x 0 x Proof The fucto fx 15 has coordate Lpschtz costats L = A λ + γ R + λ ad covexty parameter γ wth respect to the uweghted Eucldea orm The strog covexty parameter of fx wth respect to the orm L defed 3 s / R + λ = R + µ = γ Accordg to Theorem 1 we havee[d Dx 0 ] t suffces to have the umber of teratosk to be larger tha R µ logc/ǫ = + logc/ǫ = Ths fshes the proof 1 µ + R logc/ǫ + kc exp µ k C Therefore R logc/ǫ Several state-of-the-art algorthms for ERM cludg SDCA [19] SAG [15 17] ad SVRG [7 3] obta the terato complexty O log1/ǫ 17 + R We ote that our result 16 ca be much better for ll-codtoed problems e whe the codto umber R s larger tha Ths s also cofrmed by our umercal expermets Secto 4 The complexty boud 17 for the aforemetoed work s for mmzg the prmal objectve Px or the dualty gappx Dx but our result Theorem s terms of the dual optmalty I the full report [9] we show that the same guaratee o accelerated prmal-dual covergece ca be obtaed by our method wth a extra prmal gradet step wthout affectg the overall complexty The expermets Secto 4 llustrate superor performace of our algorthm o reducg the prmal objectve value eve wthout performg the extra step 6
7 We ote that Shalev-Shwartz ad Zhag [0] recetly developed a accelerated SDCA method whch acheves the same complextyo + log1/ǫ as our method Ther method calls the SDCA method a full-dmesoal accelerated gradet method a er-outer terato procedure I cotrast our APCG method s a straghtforward sgle loop coordate gradet method 31 Implemetato detals Here we show how to explot the structure of the regularzed ERM problem to effcetly compute the coordate gradet k fy k ad totally avod full-dmesoal updates Algorthm 4 We focus o the specal casegw = 1 w ad show how to compute k fy k Accordg to 15 k fy k = 1 λ A T Ayk + γ yk k Sce we do ot form y k Algorthm 4 we update Ay k by storg ad updatg two vectors R d : p k = Au k ad q k = Av k The resultg method s detaled Algorthm 5 Algorthm 5: APCG for solvg dual ERM Iput: x 0 domψ ad covexty parameter µ > 0 Italze: set α = µ 1 α ad ρ = 1+α ad let u0 = 0 v 0 = x 0 p 0 = 0 ad q 0 = Ax 0 Iterate: repeat fork = 01 1 Choose k 1} uformly at radom compute the coordate gradet k k = 1 λ ρ k+1 A T k p k +A T k q k + γ ρ k+1 u k k +v k k Compute coordate cremet } k α Ak k = argm + λ + k k + 1 φ k ρ k+1 u k k v k k R N k 3 Let u k+1 = u k ad v k+1 = v k ad update u k+1 k = u k k 1 α ρ k+1 k k p k+1 = p k 1 α ρ k+1 A k k k v k+1 k = v k k + 1+α k k q k+1 = q k + 1+α A k k k 18 Output: approxmate prmal ad dual solutos w k+1 = 1 λ ρ k+ p k+1 +q k+1 x k+1 = ρ k+1 u k+1 +v k+1 Each terato of Algorthm 5 oly volves the two er products A T k p k A T k q k computg k k ad the two vector addtos 18 They all cost Od rather tha O Whe the A s are sparse the case of most large-scale problems these operatos ca be carred out very effcetly Bascally each terato of Algorthm 5 oly costs twce as much as that of SDCA [6 19] 4 Expermets I our expermets we solve ERM problems wth smoothed hge loss for bary classfcato That s we pre-multply each feature vector A by ts label b ±1} ad use the loss fucto 0 f a 1 φa = 1 a γ f a 1 γ 1 γ 1 a otherwse The cojugate fucto ofφsφ b = b+ γ b fb [ 10] ad otherwse Therefore we have Ψ x = 1 φ x γ x x = f x [01] otherwse The dataset used our expermets are summarzed Table 1 7
8 λ rcv1 covertype ews0 AFG SDCA APCG AFG SDCA APCG AFG SDCA APCG AFG SDCA APCG Fgure 1: Comparg the APCG method wth SDCA ad the accelerated full gradet method AFG wth adaptve le search I each plot the vertcal axs s the prmal objectve gappw k P ad the horzotal axs s the umber of passes through the etre dataset The three colums correspod to the three datasets ad each row correspods to a partcular value of the regularzato parameter λ I our expermets we compare the APCG method wth SDCA ad the accelerated full gradet method AFG [1] wth a addtoal le search procedure to mprove effcecy Whe the regularzato parameter λ s ot too small aroud the APCG performs smlarly as SDCA as predcted by our complexty results ad they both outperform AFG by a substatal marg Fgure 1 shows the results the ll-codtoed settg wth λ varyg form to 10 8 Here we see that APCG has superor performace reducg the prmal objectve value compared wth SDCA ad AFG eve though our theory oly gves complexty for solvg the dual ERM problem AFG evetually catches up for cases wth very large codto umber see the plots forλ = 10 8 datasets umber of samples umber of features d sparsty rcv % covtype % ews % Table 1: Characterstcs of three bary classfcato datasets avalable from the LIBSVM web page: cjl/lbsvmtools/datasets 8
9 Refereces [1] A Beck ad M Teboulle A fast teratve shrkage-threshold algorthm for lear verse problems SIAM Joural o Imagg Sceces 1: [] A Beck ad L Tetruashvl O the covergece of block coordate descet type methods SIAM Joural o Optmzato 134: [3] K-W Chag C-J Hseh ad C-J L Coordate descet method for large-scale l -loss lear support vector maches Joural of Mache Learg Research 9: [4] O Fercoq ad P Rchtárk Accelerated parallel ad proxmal coordate descet Mauscrpt arxv: [5] M Hog X Wag M Razavyay ad Z Q Luo Iterato complexty aalyss of block coordate descet methods Mauscrpt arxv: [6] C-J Hseh K-W Chag C-J L S-S Keerth ad S Sudararaja A dual coordate descet method for large-scale lear svm I Proceedgs of the 5th Iteratoal Coferece o Mache Learg ICML pages [7] R Johso ad T Zhag Acceleratg stochastc gradet descet usg predctve varace reducto I Advaces Neural Iformato Processg Systems 6 pages [8] Y T Lee ad A Sdford Effcet accelerated coordate descet methods ad faster algorthms for solvg lear systems arxv: [9] Q L Z Lu ad L Xao A accelerated proxmal coordate gradet method ad ts applcato to regularzed emprcal rsk mmzato Techcal Report MSR-TR Mcrosoft Research 014 arxv: [10] Z Lu ad L Xao O the complexty aalyss of radomzed block-coordate descet methods Accepted by Mathematcal Programmg Seres A 014 arxv: [11] Z Q Luo ad P Tseg O the covergece of the coordate descet method for covex dfferetable mmzato Joural of Optmzato Theory & Applcatos 71: [1] Y Nesterov Itroductory Lectures o Covex Optmzato: A Basc Course Kluwer Bosto 004 [13] Y Nesterov Effcecy of coordate descet methods o huge-scale optmzato problems SIAM Joural o Optmzato : [14] P Rchtárk ad M Takáč Iterato complexty of radomzed block-coordate descet methods for mmzg a composte fucto Mathematcal Programmg 1441: [15] N Le Roux M Schmdt ad F Bach A stochastc gradet method wth a expoetal covergece rate for fte trag sets I Advaces Neural Iformato Processg Systems 5 pages [16] A Saha ad A Tewar O the o-asymptotc covergece of cyclc coordate descet methods SIAM Jorual o Optmzato 3: [17] M Schmdt N Le Roux ad F Bach Mmzg fte sums wth the stochastc average gradet Techcal Report HAL INRIA Pars Frace 013 [18] S Shalev-Shwartz ad A Tewar Stochastc methods forl 1 regularzed loss mmzato I Proceedgs of the 6th Iteratoal Coferece o Mache Learg ICML pages Motreal Caada 009 [19] S Shalev-Shwartz ad T Zhag Stochastc dual coordate ascet methods for regularzed loss mmzato Joural of Mache Learg Research 14: [0] S Shalev-Shwartz ad T Zhag Accelerated proxmal stochastc dual coordate ascet for regularzed loss mmzato Proceedgs of the 31st Iteratoal Coferece o Mache Learg ICML JMLR W&CP 31: [1] P Tseg Covergece of a block coordate descet method for odfferetable mmzato Joural of Optmzato Theory ad Applcatos 140: [] P Tseg O accelerated proxmal gradet methods for covex-cocave optmzato Upublshed mauscrpt 008 [3] L Xao ad T Zhag A proxmal stochastc gradet method wth progressve varace reducto Techcal Report MSR-TR Mcrosoft Research 014 arxv:
arxiv: v1 [cs.lg] 22 Feb 2015
SDCA wthout Dualty Sha Shalev-Shwartz arxv:50.0677v cs.lg Feb 05 Abstract Stochastc Dual Coordate Ascet s a popular method for solvg regularzed loss mmzato for the case of covex losses. I ths paper we
More informationDistributed Accelerated Proximal Coordinate Gradient Methods
Dstrbuted Accelerated Proxmal Coordate Gradet Methods Yog Re, Ju Zhu Ceter for Bo-Ispred Computg Research State Key Lab for Itell. Tech. & Systems Dept. of Comp. Sc. & Tech., TNLst Lab, Tsghua Uversty
More informationFeature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture)
CSE 546: Mache Learg Lecture 6 Feature Selecto: Part 2 Istructor: Sham Kakade Greedy Algorthms (cotued from the last lecture) There are varety of greedy algorthms ad umerous amg covetos for these algorthms.
More informationDimensionality Reduction and Learning
CMSC 35900 (Sprg 009) Large Scale Learg Lecture: 3 Dmesoalty Reducto ad Learg Istructors: Sham Kakade ad Greg Shakharovch L Supervsed Methods ad Dmesoalty Reducto The theme of these two lectures s that
More informationStochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimization
Stochastc Prmal-Dual Coordate Method for Regularzed Emprcal Rsk Mmzato Yuche Zhag L Xao September 24 Abstract We cosder a geerc covex optmzato problem assocated wth regularzed emprcal rsk mmzato of lear
More informationSolving Constrained Flow-Shop Scheduling. Problems with Three Machines
It J Cotemp Math Sceces, Vol 5, 2010, o 19, 921-929 Solvg Costraed Flow-Shop Schedulg Problems wth Three Maches P Pada ad P Rajedra Departmet of Mathematcs, School of Advaced Sceces, VIT Uversty, Vellore-632
More informationSupervised learning: Linear regression Logistic regression
CS 57 Itroducto to AI Lecture 4 Supervsed learg: Lear regresso Logstc regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Data: D { D D.. D D Supervsed learg d a set of eamples s
More informationAnalysis of Lagrange Interpolation Formula
P IJISET - Iteratoal Joural of Iovatve Scece, Egeerg & Techology, Vol. Issue, December 4. www.jset.com ISS 348 7968 Aalyss of Lagrage Iterpolato Formula Vjay Dahya PDepartmet of MathematcsMaharaja Surajmal
More informationLECTURE 24 LECTURE OUTLINE
LECTURE 24 LECTURE OUTLINE Gradet proxmal mmzato method Noquadratc proxmal algorthms Etropy mmzato algorthm Expoetal augmeted Lagraga mehod Etropc descet algorthm **************************************
More informationPart 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971))
art 4b Asymptotc Results for MRR usg RESS Recall that the RESS statstc s a specal type of cross valdato procedure (see Alle (97)) partcular to the regresso problem ad volves fdg Y $,, the estmate at the
More informationLecture 16: Backpropogation Algorithm Neural Networks with smooth activation functions
CO-511: Learg Theory prg 2017 Lecturer: Ro Lv Lecture 16: Bacpropogato Algorthm Dsclamer: These otes have ot bee subected to the usual scruty reserved for formal publcatos. They may be dstrbuted outsde
More informationResearch Article A New Iterative Method for Common Fixed Points of a Finite Family of Nonexpansive Mappings
Hdaw Publshg Corporato Iteratoal Joural of Mathematcs ad Mathematcal Sceces Volume 009, Artcle ID 391839, 9 pages do:10.1155/009/391839 Research Artcle A New Iteratve Method for Commo Fxed Pots of a Fte
More informationA Remark on the Uniform Convergence of Some Sequences of Functions
Advaces Pure Mathematcs 05 5 57-533 Publshed Ole July 05 ScRes. http://www.scrp.org/joural/apm http://dx.do.org/0.436/apm.05.59048 A Remark o the Uform Covergece of Some Sequeces of Fuctos Guy Degla Isttut
More informationInvestigation of Partially Conditional RP Model with Response Error. Ed Stanek
Partally Codtoal Radom Permutato Model 7- vestgato of Partally Codtoal RP Model wth Respose Error TRODUCTO Ed Staek We explore the predctor that wll result a smple radom sample wth respose error whe a
More informationIntroduction to local (nonparametric) density estimation. methods
Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest
More informationCommunication-Efficient Distributed Primal-Dual Algorithm for Saddle Point Problems
Commucato-Effcet Dstrbuted Prmal-Dual Algorthm for Saddle Pot Problems Yaodog Yu Nayag Techologcal Uversty ydyu@tu.edu.sg Sul Lu Nayag Techologcal Uversty lusl@tu.edu.sg So Jal Pa Nayag Techologcal Uversty
More informationSupport vector machines
CS 75 Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Outle Outle: Algorthms for lear decso boudary Support vector maches Mamum marg hyperplae.
More informationPROJECTION PROBLEM FOR REGULAR POLYGONS
Joural of Mathematcal Sceces: Advaces ad Applcatos Volume, Number, 008, Pages 95-50 PROJECTION PROBLEM FOR REGULAR POLYGONS College of Scece Bejg Forestry Uversty Bejg 0008 P. R. Cha e-mal: sl@bjfu.edu.c
More informationCS 2750 Machine Learning. Lecture 8. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x
CS 75 Mache Learg Lecture 8 Lear regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear combato of put compoets f + + + K d d K k - parameters
More informationChapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
More informationOrdinary Least Squares Regression. Simple Regression. Algebra and Assumptions.
Ordary Least Squares egresso. Smple egresso. Algebra ad Assumptos. I ths part of the course we are gog to study a techque for aalysg the lear relatoshp betwee two varables Y ad X. We have pars of observatos
More informationChapter 5 Properties of a Random Sample
Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample
More informationStrong Convergence of Weighted Averaged Approximants of Asymptotically Nonexpansive Mappings in Banach Spaces without Uniform Convexity
BULLETIN of the MALAYSIAN MATHEMATICAL SCIENCES SOCIETY Bull. Malays. Math. Sc. Soc. () 7 (004), 5 35 Strog Covergece of Weghted Averaged Appromats of Asymptotcally Noepasve Mappgs Baach Spaces wthout
More informationRandomized Dual Coordinate Ascent with Arbitrary Sampling
Radomzed Dual Coordate Ascet wth Arbtrary Samplg Zheg Qu Peter Rchtárk Tog Zhag November 21, 2014 Abstract We study the problem of mmzg the average of a large umber of smooth covex fuctos pealzed wth a
More informationRademacher Complexity. Examples
Algorthmc Foudatos of Learg Lecture 3 Rademacher Complexty. Examples Lecturer: Patrck Rebesch Verso: October 16th 018 3.1 Itroducto I the last lecture we troduced the oto of Rademacher complexty ad showed
More informationFunctions of Random Variables
Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,
More informationA New Family of Transformations for Lifetime Data
Proceedgs of the World Cogress o Egeerg 4 Vol I, WCE 4, July - 4, 4, Lodo, U.K. A New Famly of Trasformatos for Lfetme Data Lakhaa Watthaacheewakul Abstract A famly of trasformatos s the oe of several
More informationGeneralized Linear Regression with Regularization
Geeralze Lear Regresso wth Regularzato Zoya Bylsk March 3, 05 BASIC REGRESSION PROBLEM Note: I the followg otes I wll make explct what s a vector a what s a scalar usg vec t or otato, to avo cofuso betwee
More informationUnsupervised Learning and Other Neural Networks
CSE 53 Soft Computg NOT PART OF THE FINAL Usupervsed Learg ad Other Neural Networs Itroducto Mture Destes ad Idetfablty ML Estmates Applcato to Normal Mtures Other Neural Networs Itroducto Prevously, all
More information18.413: Error Correcting Codes Lab March 2, Lecture 8
18.413: Error Correctg Codes Lab March 2, 2004 Lecturer: Dael A. Spelma Lecture 8 8.1 Vector Spaces A set C {0, 1} s a vector space f for x all C ad y C, x + y C, where we take addto to be compoet wse
More information1 Review and Overview
CS9T/STATS3: Statstcal Learg Teory Lecturer: Tegyu Ma Lecture #7 Scrbe: Bra Zag October 5, 08 Revew ad Overvew We wll frst gve a bref revew of wat as bee covered so far I te frst few lectures, we stated
More informationUnimodality Tests for Global Optimization of Single Variable Functions Using Statistical Methods
Malaysa Umodalty Joural Tests of Mathematcal for Global Optmzato Sceces (): of 05 Sgle - 5 Varable (007) Fuctos Usg Statstcal Methods Umodalty Tests for Global Optmzato of Sgle Varable Fuctos Usg Statstcal
More informationMultivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
More informationLecture 3 Probability review (cont d)
STATS 00: Itroducto to Statstcal Iferece Autum 06 Lecture 3 Probablty revew (cot d) 3. Jot dstrbutos If radom varables X,..., X k are depedet, the ther dstrbuto may be specfed by specfyg the dvdual dstrbuto
More informationAn Introduction to. Support Vector Machine
A Itroducto to Support Vector Mache Support Vector Mache (SVM) A classfer derved from statstcal learg theory by Vapk, et al. 99 SVM became famous whe, usg mages as put, t gave accuracy comparable to eural-etwork
More informationKernel-based Methods and Support Vector Machines
Kerel-based Methods ad Support Vector Maches Larr Holder CptS 570 Mache Learg School of Electrcal Egeerg ad Computer Scece Washgto State Uverst Refereces Muller et al. A Itroducto to Kerel-Based Learg
More informationA tighter lower bound on the circuit size of the hardest Boolean functions
Electroc Colloquum o Computatoal Complexty, Report No. 86 2011) A tghter lower boud o the crcut sze of the hardest Boolea fuctos Masak Yamamoto Abstract I [IPL2005], Fradse ad Mlterse mproved bouds o the
More informationCS 1675 Introduction to Machine Learning Lecture 12 Support vector machines
CS 675 Itroducto to Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Mdterm eam October 9, 7 I-class eam Closed book Stud materal: Lecture otes Correspodg chapters
More informationChapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements
Aoucemets No-Parametrc Desty Estmato Techques HW assged Most of ths lecture was o the blacboard. These sldes cover the same materal as preseted DHS Bometrcs CSE 90-a Lecture 7 CSE90a Fall 06 CSE90a Fall
More informationCS286.2 Lecture 4: Dinur s Proof of the PCP Theorem
CS86. Lecture 4: Dur s Proof of the PCP Theorem Scrbe: Thom Bohdaowcz Prevously, we have prove a weak verso of the PCP theorem: NP PCP 1,1/ (r = poly, q = O(1)). Wth ths result we have the desred costat
More informationMaximum Likelihood Estimation
Marquette Uverst Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 08 b Marquette Uverst Maxmum Lkelhood Estmato We have bee sag that ~
More informationCubic Nonpolynomial Spline Approach to the Solution of a Second Order Two-Point Boundary Value Problem
Joural of Amerca Scece ;6( Cubc Nopolyomal Sple Approach to the Soluto of a Secod Order Two-Pot Boudary Value Problem W.K. Zahra, F.A. Abd El-Salam, A.A. El-Sabbagh ad Z.A. ZAk * Departmet of Egeerg athematcs
More informationPTAS for Bin-Packing
CS 663: Patter Matchg Algorthms Scrbe: Che Jag /9/00. Itroducto PTAS for B-Packg The B-Packg problem s NP-hard. If we use approxmato algorthms, the B-Packg problem could be solved polyomal tme. For example,
More informationSimple Linear Regression
Statstcal Methods I (EST 75) Page 139 Smple Lear Regresso Smple regresso applcatos are used to ft a model descrbg a lear relatoshp betwee two varables. The aspects of least squares regresso ad correlato
More informationANALYSIS ON THE NATURE OF THE BASIC EQUATIONS IN SYNERGETIC INTER-REPRESENTATION NETWORK
Far East Joural of Appled Mathematcs Volume, Number, 2008, Pages Ths paper s avalable ole at http://www.pphm.com 2008 Pushpa Publshg House ANALYSIS ON THE NATURE OF THE ASI EQUATIONS IN SYNERGETI INTER-REPRESENTATION
More informationPoint Estimation: definition of estimators
Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.
More informationBayes (Naïve or not) Classifiers: Generative Approach
Logstc regresso Bayes (Naïve or ot) Classfers: Geeratve Approach What do we mea by Geeratve approach: Lear p(y), p(x y) ad the apply bayes rule to compute p(y x) for makg predctos Ths s essetally makg
More informationESS Line Fitting
ESS 5 014 17. Le Fttg A very commo problem data aalyss s lookg for relatoshpetwee dfferet parameters ad fttg les or surfaces to data. The smplest example s fttg a straght le ad we wll dscuss that here
More informationarxiv: v1 [math.oc] 7 Mar 2017
Explotg Strog Covexty from Data wth Prmal-Dual Frst-Order Algorthms Jale Wag L Xao arxv:703.064v [math.oc] 7 Mar 07 Abstract We cosder emprcal rsk mmzato of lear predctors wth covex loss fuctos. Such problems
More informationL5 Polynomial / Spline Curves
L5 Polyomal / Sple Curves Cotets Coc sectos Polyomal Curves Hermte Curves Bezer Curves B-Sples No-Uform Ratoal B-Sples (NURBS) Mapulato ad Represetato of Curves Types of Curve Equatos Implct: Descrbe a
More informationA conic cutting surface method for linear-quadraticsemidefinite
A coc cuttg surface method for lear-quadratcsemdefte programmg Mohammad R. Osoorouch Calfora State Uversty Sa Marcos Sa Marcos, CA Jot wor wth Joh E. Mtchell RPI July 3, 2008 Outle: Secod-order coe: defto
More informationX ε ) = 0, or equivalently, lim
Revew for the prevous lecture Cocepts: order statstcs Theorems: Dstrbutos of order statstcs Examples: How to get the dstrbuto of order statstcs Chapter 5 Propertes of a Radom Sample Secto 55 Covergece
More informationCHAPTER VI Statistical Analysis of Experimental Data
Chapter VI Statstcal Aalyss of Expermetal Data CHAPTER VI Statstcal Aalyss of Expermetal Data Measuremets do ot lead to a uque value. Ths s a result of the multtude of errors (maly radom errors) that ca
More informationDifference of Convex Functions Optimization Methods: Algorithm of Minimum Maximal Network Flow Problem with Time-Windows
Iteratoal Joural of Sceces: Basc ad Appled Research (IJSBAR) ISSN 307-453 (Prt & Ole) http://gssrr.org/dex.php?oural=jouralofbascadappled ---------------------------------------------------------------------------------------------------------------------------
More informationAitken delta-squared generalized Juncgk-type iterative procedure
Atke delta-squared geeralzed Jucgk-type teratve procedure M. De la Se Isttute of Research ad Developmet of Processes. Uversty of Basque Coutry Campus of Leoa (Bzkaa) PO Box. 644- Blbao, 488- Blbao. SPAIN
More informationBinary classification: Support Vector Machines
CS 57 Itroducto to AI Lecture 6 Bar classfcato: Support Vector Maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Supervsed learg Data: D { D, D,.., D} a set of eamples D, (,,,,,
More informationBeam Warming Second-Order Upwind Method
Beam Warmg Secod-Order Upwd Method Petr Valeta Jauary 6, 015 Ths documet s a part of the assessmet work for the subject 1DRP Dfferetal Equatos o Computer lectured o FNSPE CTU Prague. Abstract Ths documet
More informationBayes Estimator for Exponential Distribution with Extension of Jeffery Prior Information
Malaysa Joural of Mathematcal Sceces (): 97- (9) Bayes Estmator for Expoetal Dstrbuto wth Exteso of Jeffery Pror Iformato Hadeel Salm Al-Kutub ad Noor Akma Ibrahm Isttute for Mathematcal Research, Uverst
More informationParallel Multi-splitting Proximal Method for Star Networks
Parallel Mult-splttg Proxmal Method for Star Networks Erm We Departmet of Electrcal Egeerg ad Computer Scece Northwester Uversty Evasto, IL 600 erm.we@orthwester.edu Abstract We develop a parallel algorthm
More informationSTRONG CONSISTENCY FOR SIMPLE LINEAR EV MODEL WITH v/ -MIXING
Joural of tatstcs: Advaces Theory ad Alcatos Volume 5, Number, 6, Pages 3- Avalable at htt://scetfcadvaces.co. DOI: htt://d.do.org/.864/jsata_7678 TRONG CONITENCY FOR IMPLE LINEAR EV MODEL WITH v/ -MIXING
More informationTESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
More informationOverview. Basic concepts of Bayesian learning. Most probable model given data Coin tosses Linear regression Logistic regression
Overvew Basc cocepts of Bayesa learg Most probable model gve data Co tosses Lear regresso Logstc regresso Bayesa predctos Co tosses Lear regresso 30 Recap: regresso problems Iput to learg problem: trag
More informationEconometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
More informationCHAPTER 4 RADICAL EXPRESSIONS
6 CHAPTER RADICAL EXPRESSIONS. The th Root of a Real Number A real umber a s called the th root of a real umber b f Thus, for example: s a square root of sce. s also a square root of sce ( ). s a cube
More informationEstimation of Stress- Strength Reliability model using finite mixture of exponential distributions
Iteratoal Joural of Computatoal Egeerg Research Vol, 0 Issue, Estmato of Stress- Stregth Relablty model usg fte mxture of expoetal dstrbutos K.Sadhya, T.S.Umamaheswar Departmet of Mathematcs, Lal Bhadur
More informationLecture 3. Sampling, sampling distributions, and parameter estimation
Lecture 3 Samplg, samplg dstrbutos, ad parameter estmato Samplg Defto Populato s defed as the collecto of all the possble observatos of terest. The collecto of observatos we take from the populato s called
More information18.657: Mathematics of Machine Learning
8.657: Mathematcs of Mache Learg Lecturer: Phlppe Rgollet Lecture 3 Scrbe: James Hrst Sep. 6, 205.5 Learg wth a fte dctoary Recall from the ed of last lecture our setup: We are workg wth a fte dctoary
More informationRuntime analysis RLS on OneMax. Heuristic Optimization
Lecture 6 Rutme aalyss RLS o OeMax trals of {,, },, l ( + ɛ) l ( ɛ)( ) l Algorthm Egeerg Group Hasso Platter Isttute, Uversty of Potsdam 9 May T, We wat to rgorously uderstad ths behavor 9 May / Rutme
More informationCIS 800/002 The Algorithmic Foundations of Data Privacy October 13, Lecture 9. Database Update Algorithms: Multiplicative Weights
CIS 800/002 The Algorthmc Foudatos of Data Prvacy October 13, 2011 Lecturer: Aaro Roth Lecture 9 Scrbe: Aaro Roth Database Update Algorthms: Multplcatve Weghts We ll recall aga) some deftos from last tme:
More informationAnalyzing Fuzzy System Reliability Using Vague Set Theory
Iteratoal Joural of Appled Scece ad Egeerg 2003., : 82-88 Aalyzg Fuzzy System Relablty sg Vague Set Theory Shy-Mg Che Departmet of Computer Scece ad Iformato Egeerg, Natoal Tawa versty of Scece ad Techology,
More information4 Inner Product Spaces
11.MH1 LINEAR ALGEBRA Summary Notes 4 Ier Product Spaces Ier product s the abstracto to geeral vector spaces of the famlar dea of the scalar product of two vectors or 3. I what follows, keep these key
More informationSimulation Output Analysis
Smulato Output Aalyss Summary Examples Parameter Estmato Sample Mea ad Varace Pot ad Iterval Estmato ermatg ad o-ermatg Smulato Mea Square Errors Example: Sgle Server Queueg System x(t) S 4 S 4 S 3 S 5
More informationObjectives of Multiple Regression
Obectves of Multple Regresso Establsh the lear equato that best predcts values of a depedet varable Y usg more tha oe eplaator varable from a large set of potetal predctors {,,... k }. Fd that subset of
More informationA Robust Total Least Mean Square Algorithm For Nonlinear Adaptive Filter
A Robust otal east Mea Square Algorthm For Nolear Adaptve Flter Ruxua We School of Electroc ad Iformato Egeerg X'a Jaotog Uversty X'a 70049, P.R. Cha rxwe@chare.com Chogzhao Ha, azhe u School of Electroc
More informationSupport vector machines II
CS 75 Mache Learg Lecture Support vector maches II Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Learl separable classes Learl separable classes: here s a hperplae that separates trag staces th o error
More information10.1 Approximation Algorithms
290 0. Approxmato Algorthms Let us exame a problem, where we are gve A groud set U wth m elemets A collecto of subsets of the groud set = {,, } s.t. t s a cover of U: = U The am s to fd a subcover, = U,
More informationSTATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS. x, where. = y - ˆ " 1
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumpto E(Y x) η 0 + η x (lear codtoal mea fucto) Data (x, y ), (x 2, y 2 ),, (x, y ) Least squares estmator ˆ E (Y x) ˆ " 0 + ˆ " x, where ˆ
More informationQR Factorization and Singular Value Decomposition COS 323
QR Factorzato ad Sgular Value Decomposto COS 33 Why Yet Aother Method? How do we solve least-squares wthout currg codto-squarg effect of ormal equatos (A T A A T b) whe A s sgular, fat, or otherwse poorly-specfed?
More informationå 1 13 Practice Final Examination Solutions - = CS109 Dec 5, 2018
Chrs Pech Fal Practce CS09 Dec 5, 08 Practce Fal Examato Solutos. Aswer: 4/5 8/7. There are multle ways to obta ths aswer; here are two: The frst commo method s to sum over all ossbltes for the rak of
More informationNon-uniform Turán-type problems
Joural of Combatoral Theory, Seres A 111 2005 106 110 wwwelsevercomlocatecta No-uform Turá-type problems DhruvMubay 1, Y Zhao 2 Departmet of Mathematcs, Statstcs, ad Computer Scece, Uversty of Illos at
More informationLecture Note to Rice Chapter 8
ECON 430 HG revsed Nov 06 Lecture Note to Rce Chapter 8 Radom matrces Let Y, =,,, m, =,,, be radom varables (r.v. s). The matrx Y Y Y Y Y Y Y Y Y Y = m m m s called a radom matrx ( wth a ot m-dmesoal dstrbuto,
More informationMATH 247/Winter Notes on the adjoint and on normal operators.
MATH 47/Wter 00 Notes o the adjot ad o ormal operators I these otes, V s a fte dmesoal er product space over, wth gve er * product uv, T, S, T, are lear operators o V U, W are subspaces of V Whe we say
More informationQ-analogue of a Linear Transformation Preserving Log-concavity
Iteratoal Joural of Algebra, Vol. 1, 2007, o. 2, 87-94 Q-aalogue of a Lear Trasformato Preservg Log-cocavty Daozhog Luo Departmet of Mathematcs, Huaqao Uversty Quazhou, Fua 362021, P. R. Cha ldzblue@163.com
More informationLikewise, properties of the optimal policy for equipment replacement & maintenance problems can be used to reduce the computation.
Whe solvg a vetory repleshmet problem usg a MDP model, kowg that the optmal polcy s of the form (s,s) ca reduce the computatoal burde. That s, f t s optmal to replesh the vetory whe the vetory level s,
More informationEntropy ISSN by MDPI
Etropy 2003, 5, 233-238 Etropy ISSN 1099-4300 2003 by MDPI www.mdp.org/etropy O the Measure Etropy of Addtve Cellular Automata Hasa Aı Arts ad Sceces Faculty, Departmet of Mathematcs, Harra Uversty; 63100,
More information( ) 2 2. Multi-Layer Refraction Problem Rafael Espericueta, Bakersfield College, November, 2006
Mult-Layer Refracto Problem Rafael Espercueta, Bakersfeld College, November, 006 Lght travels at dfferet speeds through dfferet meda, but refracts at layer boudares order to traverse the least-tme path.
More informationLecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model
Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The
More informationρ < 1 be five real numbers. The
Lecture o BST 63: Statstcal Theory I Ku Zhag, /0/006 Revew for the prevous lecture Deftos: covarace, correlato Examples: How to calculate covarace ad correlato Theorems: propertes of correlato ad covarace
More information6.867 Machine Learning
6.867 Mache Learg Problem set Due Frday, September 9, rectato Please address all questos ad commets about ths problem set to 6.867-staff@a.mt.edu. You do ot eed to use MATLAB for ths problem set though
More informationCOMPROMISE HYPERSPHERE FOR STOCHASTIC DOMINANCE MODEL
Sebasta Starz COMPROMISE HYPERSPHERE FOR STOCHASTIC DOMINANCE MODEL Abstract The am of the work s to preset a method of rakg a fte set of dscrete radom varables. The proposed method s based o two approaches:
More informationLecture 9: Tolerant Testing
Lecture 9: Tolerat Testg Dael Kae Scrbe: Sakeerth Rao Aprl 4, 07 Abstract I ths lecture we prove a quas lear lower boud o the umber of samples eeded to do tolerat testg for L dstace. Tolerat Testg We have
More informationThe Mathematical Appendix
The Mathematcal Appedx Defto A: If ( Λ, Ω, where ( λ λ λ whch the probablty dstrbutos,,..., Defto A. uppose that ( Λ,,..., s a expermet type, the σ-algebra o λ λ λ are defed s deoted by ( (,,...,, σ Ω.
More informationLinear Regression Linear Regression with Shrinkage. Some slides are due to Tommi Jaakkola, MIT AI Lab
Lear Regresso Lear Regresso th Shrkage Some sldes are due to Tomm Jaakkola, MIT AI Lab Itroducto The goal of regresso s to make quattatve real valued predctos o the bass of a vector of features or attrbutes.
More informationIntroduction to Matrices and Matrix Approach to Simple Linear Regression
Itroducto to Matrces ad Matrx Approach to Smple Lear Regresso Matrces Defto: A matrx s a rectagular array of umbers or symbolc elemets I may applcatos, the rows of a matrx wll represet dvduals cases (people,
More informationMA/CSSE 473 Day 27. Dynamic programming
MA/CSSE 473 Day 7 Dyamc Programmg Bomal Coeffcets Warshall's algorthm (Optmal BSTs) Studet questos? Dyamc programmg Used for problems wth recursve solutos ad overlappg subproblems Typcally, we save (memoze)
More information( ) = ( ) ( ) Chapter 13 Asymptotic Theory and Stochastic Regressors. Stochastic regressors model
Chapter 3 Asmptotc Theor ad Stochastc Regressors The ature of eplaator varable s assumed to be o-stochastc or fed repeated samples a regresso aalss Such a assumpto s approprate for those epermets whch
More informationAnalysis of Variance with Weibull Data
Aalyss of Varace wth Webull Data Lahaa Watthaacheewaul Abstract I statstcal data aalyss by aalyss of varace, the usual basc assumptos are that the model s addtve ad the errors are radomly, depedetly, ad
More informationECONOMETRIC THEORY. MODULE VIII Lecture - 26 Heteroskedasticity
ECONOMETRIC THEORY MODULE VIII Lecture - 6 Heteroskedastcty Dr. Shalabh Departmet of Mathematcs ad Statstcs Ida Isttute of Techology Kapur . Breusch Paga test Ths test ca be appled whe the replcated data
More informationInvestigating Cellular Automata
Researcher: Taylor Dupuy Advsor: Aaro Wootto Semester: Fall 4 Ivestgatg Cellular Automata A Overvew of Cellular Automata: Cellular Automata are smple computer programs that geerate rows of black ad whte
More informationCSE 5526: Introduction to Neural Networks Linear Regression
CSE 556: Itroducto to Neural Netorks Lear Regresso Part II 1 Problem statemet Part II Problem statemet Part II 3 Lear regresso th oe varable Gve a set of N pars of data , appromate d by a lear fucto
More information