A new type of optimization method based on conjugate directions
|
|
- Ella Cobb
- 5 years ago
- Views:
Transcription
1 A ew type of optmzato method based o cojugate drectos Pa X Scece School aj Uversty of echology ad Educato (UE aj Cha e-mal: pax94@sacom Abstract A ew type of optmzato method based o cojugate drectos s proposed ths paper It ca be proved that ths type of method has quadratc termato property wthout exact le search he ew method requres oly the storage of 4 vectors such that t s sutable for large scale problems Numercal expereces show that the ew method s effectve Keywords-optmzato; large scale problems; cojugate drectos; quadratc termato property I INRODUCION Optmzato s a mportat tool decso scece aalyss of physcal systems ad cotrol egeerg Cojugate gradet (CG methods are some of the most useful algorthms for ucostraed optmzato problem m f ( x x R ( by a sequece of le searches x x αd ( from a user suppled estmate x he step legth α ca be obtaed by the le search methods Cojugate gradet algorthms are wdely used for the two otable propertes: Frst they eed ot store ay matrx thus are sutable for large scale problems; Secod whe exact le search (ELS s tae CG algorthms have the quadratc termato property whch reflects the effcecy of the algorthms he search drectos of CG methods are geerated as follows d g d ( where g f( x d gad s geerated by the defto of cojugated drectos d G d (4 where G s the Hessa of f at the pot x Varous formulas for are gve for dfferet CG algorthms[][][] g g (5 g g g ( g g (6 g g g ( g g (7 d ( g g I practce a exact le search (ELS s ot usually possble for t requres too may evaluatos of objectve fucto f ad possbly the gradet f More practcal strateges perform a exact le search (ILS to detfy a step legth However for a geeral quadratc fucto CG methods ca t guaratee global covergece whe perform ILS[4] Eve for a postve quadratc fucto whe perform ILS CG methods wll lose the quadratc termato property Example : Cosder a -dmesoal postve quadratc fucto f ( x x Ax b x whch gradet s g( x Ax b Suppose that { d d} s a set of ozero cojugate vectors Case : Gve a startg pot x let us geerate the sequece { x * } gve explctly by by Eq( where α s obtaed by ELS g d α (8 d Ad hus x x αd x x αd d d Sce g( x s lear ad d d are cojugate we have ( α d Ad d d So ad x s the mmzer of the fucto f ( x hs llustrates the cojugate drectos are good search drectos whe perform ELS Case : Startg from the same x let us perform ILS alog the search drecto d to get the step legth α α ad mproved x x α d he startg from the same x perform ELS alog d to get the step legth α ad x x αd By the property of ELS we have d But x s ot the mmzer of f ( x for ( α ( g( x d g( x Ad d g( x A( x x d (9 d A( x x Case llustrates that whe perform ILS CG methods wll lose the quadratc termato property However some Publshed by Atlats Press Pars Frace the authors
2 modfcato ca be tae o the secod search drecto Let p d γ d ( be the search drecto the secod step ad α be the step legth x x α p the g( x d g( x d αd A( d γd ( ( d A x x αγ d Ad If we correctly select α ad γ we ca get d ad d ad x s the mmzer hs llustrates that whe perform ILS f some modfcato s tae o the curret cojugate drecto we wll get a better search drecto he ew method ths paper s based o ths truth II DERIVAION OF HE NEW MEHOD Cosder -dmesoal postve quadratc fucto f ( x x Ax b x c ( Suppose p s the th search drecto x s the pot after the th le search he aylor expaso of f ( x ca be wrte as f ( x f( x g p p G p m( p (4 where p x x G A he best search drecto th step s p arg m mp ( It s dffcult to get the best drecto R space f the Hessa s uow We ca tae the dea of CG methods that uses lear combato of two lear depedet vectors to approxmate p Let accessoral vector d cojugate wth p thus d Ap the from the ht of Example the search drecto th step ca be wrtte as p ap ad d ca be geerated form FRCG or PRCG as d g p (5 Obvously f a p s cojugate gradet drecto We amed the ew method Modfed Cojugate Method (MCG he ey to obta MCG search drecto s the selecto of combato coeffcets a ad a whch s smlar to the dea of Yua [5] LetU [ p d ] y [ a a] he p Uy (6 mp ( f( x g p p G p (7 f ( x ( U g y y U G Uy ϕ( y Let gˆ U g ˆ G U G U he we ca rewrte If y ϕ ( y f( x ˆ ˆ g y y Gy (8 arg m ϕ( y s obtaedwe ca get the mmzer of mpo ( the mafold { p ap ad a R a R} to approxmate p wrtte as p Uy p s the search drecto of MCG If G s postvethe Ĝ s also postve So we ca get p Uy (9 where y Gˆ gˆ If Ĝ s ot postvewe ca tae p ( sg g d d ( he algorthm s as follows Step : Gve x tolerace ε set ; Step : Calculate g f( x f g ε stop ad x s the mmzer; Otherwse go to step ; Step : If set p g go to step 6;Otherwse g ( g g set d g p where ; g g Step 4: Calculate s p Gp t d Gd ; Step 5: If t the p sg( d g d; Else f s the p ( sg p g p ; p g d g Otherwse p p dgo to step 6; s t Step 6: Searchα whch satsfes Wolfe codto: f ( x α p f( x cα f( x p f( x α p p c f( x p < c < c < Step 7: Set x x αp go to step Remar: Usually α s always tred frst whether t satsfes Wolfe codto whe performg ELS hat s true MCG III PROPERIES OF MCG heorem he search drectos geerated by MCG are descet drectos Proof: For the drecto geerated by Eq( p g sg( g d g d < For the drecto geerated by Eq(9 p g y U g ˆ g UG U g ( g p ( g d < p G p d G d So the search drectos geerated by Eq(9 ad Eq( are descet drectos Publshed by Atlats Press Pars Frace the authors
3 Lemma Suppose that MCG s used to solve -dmesoal postve quadratc problem defed by Eq( the teratve pots are x x ad search drectos geerated by MCG are p p p the { p p p } s a set of ozero cojugate vectors ad for x s the mmzer of the objectve fucto o the mafold { x αp α R} Proof: Sce the objectve fucto s postve quadratc form Eq(8 ad Eq(9 we ca coclude that whe usg MCG the step legth α wll be for For p Uywhere y arg m f( x U y U g x Uy p g y U g We have ( e U g hus It shows that x s the mmzer o { x αp α R} whch meas that x s obtaed by performg ELS alog p from x p U y where y arg m f( x U y p g s d g d g t t It shows that p d We ca coclude that { } p d d s a set of ozero cojugate vectors from Eq(5 ad p d herefore { } p p p s a set of ozero cojugate vectors Lemma Suppose that f ( x s a -dmesoal postve quadratc fucto { d d d } s a set of ozero cojugate vectors Let x R be ay startg pot f ELS s performed alog d d d to get the sequece x x x the x s the mmzer of f ( x [6] heorem MCG has the quadratc termato property Proof: From Lemma we ow that for postve quadratc fucto MCG equals to the cojugate gradet methods startg from the secod teratve pot whch performg ELS Combed wth Lemma t ca be cocluded that MCG has the quadratc termato property MCG ca be vew as a d of Iterated-subspace mmzato methods [7] he frame wor of Iteratedsubspace mmzato methods (ISM s as follows Step : If the th teratve pot x satsfes the toleracestop ad output x ; Otherwse cotue; t Step : Costruct Z R ad solve t -dmesoal subproblem y arg m f( x Z y ; t Step : x x Zy go to step MCG s a d of specal Iterated-subspace mmzato method whch Z [ p d] t ad Newto method s used to solve the -dmesoal sub-problems So the covergece of MCG s based o the covergece of Iterated-subspace mmzato methods whch show as follows [8] heorem Suppose that f ( x s a dfferetable fucto the level set L { x R f( x f( x } s bouded f ( x s Lpschtz-cotuous ad for some ε > < α < the sequece of pots geerated by the ISM satsfes Goldste codto f ( x g( x Zdz f( x f( x αg( x Zdz ( ad gx ( Zd z ε ( Z g( x d z where d s the soluto of subproblem m f ( x Z d z t d z R z the the ISM algorthm globally coverges to a statoary pot for problem( from ay startg pot Co [7] proofed that f egatve gradet vector s the subspace spaed by Z ad whe performg ILS wth Wolfe codto sub-problems ISM wll satsfes Eq( ad Eq( thus covergece ca be guarateed Obvously MCG satsfes the codtos above IV IMPLEMENAION OF MCG MEHOD he ey to mplemet MCG method s to calculate s p Gp ad t d Gd For the Hessa G s ot easy to calculate ad store t s reasoable to calculate s ad t approxmately We ca use the approxmato: g( x hp g( x Gp h ( g( x hd g( x Gd h (4 8 for some small dfferecg terval h h s farly close to optmal terval value[6] he s ad t ca be calculated he algorthm usg ths approxmato s amed MCG o reduce evaluatos of gradet aother approxmato ca be used: Gp α (5 where α s step legth he algorthm usg ths approxmato s amed MCG Publshed by Atlats Press Pars Frace the authors 4
4 he algorthms MCG MCG ad PRCG are mplemeted by Matlab software wth the same stoppg 5 crtero < he test fuctos are as follows F:Exteded Rose-broc / f( x ( x x ( x * [ ] f x x ( F:rda f( x ( x x x ( F:Power f( x x * x [ ] f( x F4:Exteded Beale [5 x ( x ] / f( x [5 x ( x] [65 x ( x] x [ ] F5:Noda f ( x ( x x ( x x - * [ ] f( x he umercal results of MCG MCG ad PRCG are show ABLE I he results are form of the umber of fucto calls Nf the umber of gradet calls Ng the umber of teratos NI the cetral processor ut tme CPU the orm of gradet at mmzer Norm(g ad the mmum of the fucto Fvl he results show that both MCG ad MCG outperform PRCG Ether MCG or MCG eeds o more tha 4 -dmesoal vectors to mplemet hus they are sutable for large scale problems such as -dmesoal problems or eve hgher dmesoal problems Comparg wth MCG MCG eeds less fucto calls but more gradet calls herefore whe evaluato of the objectve fucto s much easer tha ts gradet MCG s the prorty opto V CONCLUSIONS he ew optmzato method amed MCG preseted ths paper ca be vew as a d of Iterated-subspace mmzato methods whch global covergece s guarateed It has the quadratc termato property wthout ELS Numercal expermets show that MCG s more effcet ad precse tha PRCG Furthermore MCG s also sutable for large scale problems ACKNOWLEDGMEN hs wor s supported by NSFC (No 5975 No he foudatos support s greatly apprecated REFERENCES [] Fletcher R ad Reeves CM Fucto Mmzato by Cojugate Gradets[J] Computer Joural : [] Soreso H W Comparso of some cojugate drecto procedures for fucto mmzato [J] Joural of the Fral Isttute :4-44 [] Lu Y Storey C Effcet geeralzed cojugate gradet algorthms[j] Joural of optmzato theory ad applcatos 9969: 9-5 [4] Da Yad Yua Y Nolear Cojugate Gradet Methods [M] Shagha: Shagha Scece ad echology Press [5] Yua Y ad Stoer J A subspace study o cojugate algorthms[j] ZAMM Z agew Math Meth (:69-77 [6] Nocedal J ad Wrght S J Numercal Optmzato[M] Sprger Seres Operatos Research New Yor: Sprger-Verlag 999 [7] [o A R Gould N L M ad Sarteaer Aetal O Iterated- Subspace mmzato methods for olear optmzato[a] L Adams ad L Nazareth Lear ad Nolear cojugate gradet related methods[c] AMS-IMS-SIAM 996:5-78 [8] Des J E ad Schabel RB Numercal methods for ucostraed optmzato ad olear equatos[m] Pretce-Hall Eglewood Clffs NJ 98 ABLE I Performace of MCG MCG ad PRCG Fucto Dmeso Algorthm Nf Ng NI CPU Norm(g Fvl MCG E-7 45E-7 MCG E-7 59E-7 F PRCG E-7 54E- MCG E-7 6E-6 MCG E-7 59E-6 PRCG E-6 59E- MCG E-6 7E- F MCG E-6 E- PRCG E-6 549E- MCG E-6 5E-4 Publshed by Atlats Press Pars Frace the authors 5
5 F F4 F5 MCG E-6 5E-4 PRCG E-6 64E- MCG E-6 947E- MCG E-6 947E- PRCG E-6 76E- MCG E-6 75E- MCG E-6 75E- PRCG E-6 67E- MCG E-6 6E- MCG 6 9 8E-7 54E-6 PRCG E-6 95E- MCG E-6 64E- MCG E-7 8E-5 PRCG E-6 94E- MCG E-7 44E-8 MCG E-8 69E-9 PRCG E-6 6E- MCG E-7 4E-9 MCG E-8 79E- PRCG E-6 79E-6 Publshed by Atlats Press Pars Frace the authors 6
UNIT 2 SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS
Numercal Computg -I UNIT SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS Structure Page Nos..0 Itroducto 6. Objectves 7. Ital Approxmato to a Root 7. Bsecto Method 8.. Error Aalyss 9.4 Regula Fals Method
More informationSolving Constrained Flow-Shop Scheduling. Problems with Three Machines
It J Cotemp Math Sceces, Vol 5, 2010, o 19, 921-929 Solvg Costraed Flow-Shop Schedulg Problems wth Three Maches P Pada ad P Rajedra Departmet of Mathematcs, School of Advaced Sceces, VIT Uversty, Vellore-632
More informationCubic Nonpolynomial Spline Approach to the Solution of a Second Order Two-Point Boundary Value Problem
Joural of Amerca Scece ;6( Cubc Nopolyomal Sple Approach to the Soluto of a Secod Order Two-Pot Boudary Value Problem W.K. Zahra, F.A. Abd El-Salam, A.A. El-Sabbagh ad Z.A. ZAk * Departmet of Egeerg athematcs
More informationFunctions of Random Variables
Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,
More informationG S Power Flow Solution
G S Power Flow Soluto P Q I y y * 0 1, Y y Y 0 y Y Y 1, P Q ( k) ( k) * ( k 1) 1, Y Y PQ buses * 1 P Q Y ( k1) *( k) ( k) Q Im[ Y ] 1 P buses & Slack bus ( k 1) *( k) ( k) Y 1 P Re[ ] Slack bus 17 Calculato
More informationJacobian-Free Diagonal Newton s Method for Solving Nonlinear Systems with Singular Jacobian ABSTRACT INTRODUCTION
Malaysa Joural of Mathematcal Sceces 5(: 4-55 ( Jacoba-ree Dagoal Newto s Method for Solvg Nolear Systems wth Sgular Jacoba Mohammed Wazr Yusuf,, Leog Wah Jue ad, Mal Abu Hassa aculty of Scece, Uverst
More information1 0, x? x x. 1 Root finding. 1.1 Introduction. Solve[x^2-1 0,x] {{x -1},{x 1}} Plot[x^2-1,{x,-2,2}] 3
Adrew Powuk - http://www.powuk.com- Math 49 (Numercal Aalyss) Root fdg. Itroducto f ( ),?,? Solve[^-,] {{-},{}} Plot[^-,{,-,}] Cubc equato https://e.wkpeda.org/wk/cubc_fucto Quartc equato https://e.wkpeda.org/wk/quartc_fucto
More informationOn the Interval Zoro Symmetric Single Step. Procedure IZSS1-5D for the Simultaneous. Bounding of Real Polynomial Zeros
It. Joural of Math. Aalyss, Vol. 7, 2013, o. 59, 2947-2951 HIKARI Ltd, www.m-hkar.com http://dx.do.org/10.12988/ma.2013.310259 O the Iterval Zoro Symmetrc Sgle Step Procedure IZSS1-5D for the Smultaeous
More information1 Lyapunov Stability Theory
Lyapuov Stablty heory I ths secto we cosder proofs of stablty of equlbra of autoomous systems. hs s stadard theory for olear systems, ad oe of the most mportat tools the aalyss of olear systems. It may
More informationAnalysis of Lagrange Interpolation Formula
P IJISET - Iteratoal Joural of Iovatve Scece, Egeerg & Techology, Vol. Issue, December 4. www.jset.com ISS 348 7968 Aalyss of Lagrage Iterpolato Formula Vjay Dahya PDepartmet of MathematcsMaharaja Surajmal
More informationA conic cutting surface method for linear-quadraticsemidefinite
A coc cuttg surface method for lear-quadratcsemdefte programmg Mohammad R. Osoorouch Calfora State Uversty Sa Marcos Sa Marcos, CA Jot wor wth Joh E. Mtchell RPI July 3, 2008 Outle: Secod-order coe: defto
More informationFeature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture)
CSE 546: Mache Learg Lecture 6 Feature Selecto: Part 2 Istructor: Sham Kakade Greedy Algorthms (cotued from the last lecture) There are varety of greedy algorthms ad umerous amg covetos for these algorthms.
More informationBERNSTEIN COLLOCATION METHOD FOR SOLVING NONLINEAR DIFFERENTIAL EQUATIONS. Aysegul Akyuz Dascioglu and Nese Isler
Mathematcal ad Computatoal Applcatos, Vol. 8, No. 3, pp. 293-300, 203 BERNSTEIN COLLOCATION METHOD FOR SOLVING NONLINEAR DIFFERENTIAL EQUATIONS Aysegul Ayuz Dascoglu ad Nese Isler Departmet of Mathematcs,
More informationNumerical Analysis Formulae Booklet
Numercal Aalyss Formulae Booklet. Iteratve Scemes for Systems of Lear Algebrac Equatos:.... Taylor Seres... 3. Fte Dfferece Approxmatos... 3 4. Egevalues ad Egevectors of Matrces.... 3 5. Vector ad Matrx
More informationAssignment 5/MATH 247/Winter Due: Friday, February 19 in class (!) (answers will be posted right after class)
Assgmet 5/MATH 7/Wter 00 Due: Frday, February 9 class (!) (aswers wll be posted rght after class) As usual, there are peces of text, before the questos [], [], themselves. Recall: For the quadratc form
More informationMinimization of Unconstrained Nonpolynomial Large-Scale Optimization Problems Using Conjugate Gradient Method Via Exact Line Search
Amerca Joural of Mechacal ad Materals Eeer 207; (): 0-4 http://wwwscecepublshroupcom/j/ajmme do: 0648/jajmme207003 Mmzato of Ucostraed Nopolyomal Lare-Scale Optmzato Problems Us Cojuate Gradet Method Va
More informationCHAPTER 4 RADICAL EXPRESSIONS
6 CHAPTER RADICAL EXPRESSIONS. The th Root of a Real Number A real umber a s called the th root of a real umber b f Thus, for example: s a square root of sce. s also a square root of sce ( ). s a cube
More informationto the estimation of total sensitivity indices
Applcato of the cotrol o varate ate techque to the estmato of total sestvty dces S KUCHERENKO B DELPUECH Imperal College Lodo (UK) skuchereko@mperalacuk B IOOSS Electrcté de Frace (Frace) S TARANTOLA Jot
More informationPTAS for Bin-Packing
CS 663: Patter Matchg Algorthms Scrbe: Che Jag /9/00. Itroducto PTAS for B-Packg The B-Packg problem s NP-hard. If we use approxmato algorthms, the B-Packg problem could be solved polyomal tme. For example,
More informationEconometric Methods. Review of Estimation
Ecoometrc Methods Revew of Estmato Estmatg the populato mea Radom samplg Pot ad terval estmators Lear estmators Ubased estmators Lear Ubased Estmators (LUEs) Effcecy (mmum varace) ad Best Lear Ubased Estmators
More informationX ε ) = 0, or equivalently, lim
Revew for the prevous lecture Cocepts: order statstcs Theorems: Dstrbutos of order statstcs Examples: How to get the dstrbuto of order statstcs Chapter 5 Propertes of a Radom Sample Secto 55 Covergece
More informationCSE 5526: Introduction to Neural Networks Linear Regression
CSE 556: Itroducto to Neural Netorks Lear Regresso Part II 1 Problem statemet Part II Problem statemet Part II 3 Lear regresso th oe varable Gve a set of N pars of data , appromate d by a lear fucto
More informationLECTURE 24 LECTURE OUTLINE
LECTURE 24 LECTURE OUTLINE Gradet proxmal mmzato method Noquadratc proxmal algorthms Etropy mmzato algorthm Expoetal augmeted Lagraga mehod Etropc descet algorthm **************************************
More informationPoint Estimation: definition of estimators
Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.
More informationCS 1675 Introduction to Machine Learning Lecture 12 Support vector machines
CS 675 Itroducto to Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Mdterm eam October 9, 7 I-class eam Closed book Stud materal: Lecture otes Correspodg chapters
More information{ }{ ( )} (, ) = ( ) ( ) ( ) Chapter 14 Exercises in Sampling Theory. Exercise 1 (Simple random sampling): Solution:
Chapter 4 Exercses Samplg Theory Exercse (Smple radom samplg: Let there be two correlated radom varables X ad A sample of sze s draw from a populato by smple radom samplg wthout replacemet The observed
More informationSupport vector machines
CS 75 Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Outle Outle: Algorthms for lear decso boudary Support vector maches Mamum marg hyperplae.
More informationA Multistep Broyden s-type Method for Solving System of Nonlinear Equations
Joural of Novel Appled Sceces Avalable ole at www.jasc.org 7 JNAS Joural-7-6-/39-5 ISSN 3-549 7 JNAS A Multstep royde s-ype Method for Solvg System of Nolear Equatos M.Y.Wazr, M.A. Alyu ad A.Wal 3 - Departmet
More informationMULTIDIMENSIONAL HETEROGENEOUS VARIABLE PREDICTION BASED ON EXPERTS STATEMENTS. Gennadiy Lbov, Maxim Gerasimov
Iteratoal Boo Seres "Iformato Scece ad Computg" 97 MULTIIMNSIONAL HTROGNOUS VARIABL PRICTION BAS ON PRTS STATMNTS Geady Lbov Maxm Gerasmov Abstract: I the wors [ ] we proposed a approach of formg a cosesus
More informationA Penalty Function Algorithm with Objective Parameters and Constraint Penalty Parameter for Multi-Objective Programming
Aerca Joural of Operatos Research, 4, 4, 33-339 Publshed Ole Noveber 4 ScRes http://wwwscrporg/oural/aor http://ddoorg/436/aor4463 A Pealty Fucto Algorth wth Obectve Paraeters ad Costrat Pealty Paraeter
More informationUnimodality Tests for Global Optimization of Single Variable Functions Using Statistical Methods
Malaysa Umodalty Joural Tests of Mathematcal for Global Optmzato Sceces (): of 05 Sgle - 5 Varable (007) Fuctos Usg Statstcal Methods Umodalty Tests for Global Optmzato of Sgle Varable Fuctos Usg Statstcal
More informationChapter 5 Properties of a Random Sample
Lecture 6 o BST 63: Statstcal Theory I Ku Zhag, /0/008 Revew for the prevous lecture Cocepts: t-dstrbuto, F-dstrbuto Theorems: Dstrbutos of sample mea ad sample varace, relatoshp betwee sample mea ad sample
More informationA nonsmooth Levenberg-Marquardt method for generalized complementarity problem
ISSN 746-7659 Egla UK Joural of Iformato a Computg Scece Vol. 7 No. 4 0 pp. 67-7 A osmooth Leveberg-Marquart metho for geeralze complemetarty problem Shou-qag Du College of Mathematcs Qgao Uversty Qgao
More informationGeneralized Convex Functions on Fractal Sets and Two Related Inequalities
Geeralzed Covex Fuctos o Fractal Sets ad Two Related Iequaltes Huxa Mo, X Su ad Dogya Yu 3,,3School of Scece, Bejg Uversty of Posts ad Telecommucatos, Bejg,00876, Cha, Correspodece should be addressed
More informationAn Introduction to. Support Vector Machine
A Itroducto to Support Vector Mache Support Vector Mache (SVM) A classfer derved from statstcal learg theory by Vapk, et al. 99 SVM became famous whe, usg mages as put, t gave accuracy comparable to eural-etwork
More informationSupport vector machines II
CS 75 Mache Learg Lecture Support vector maches II Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Learl separable classes Learl separable classes: here s a hperplae that separates trag staces th o error
More informationSupervised learning: Linear regression Logistic regression
CS 57 Itroducto to AI Lecture 4 Supervsed learg: Lear regresso Logstc regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Data: D { D D.. D D Supervsed learg d a set of eamples s
More informationComplete Convergence and Some Maximal Inequalities for Weighted Sums of Random Variables
Joural of Sceces, Islamc Republc of Ira 8(4): -6 (007) Uversty of Tehra, ISSN 06-04 http://sceces.ut.ac.r Complete Covergece ad Some Maxmal Iequaltes for Weghted Sums of Radom Varables M. Am,,* H.R. Nl
More informationSpecial Instructions / Useful Data
JAM 6 Set of all real umbers P A..d. B, p Posso Specal Istructos / Useful Data x,, :,,, x x Probablty of a evet A Idepedetly ad detcally dstrbuted Bomal dstrbuto wth parameters ad p Posso dstrbuto wth
More informationA Robust Total Least Mean Square Algorithm For Nonlinear Adaptive Filter
A Robust otal east Mea Square Algorthm For Nolear Adaptve Flter Ruxua We School of Electroc ad Iformato Egeerg X'a Jaotog Uversty X'a 70049, P.R. Cha rxwe@chare.com Chogzhao Ha, azhe u School of Electroc
More informationThe Mathematical Appendix
The Mathematcal Appedx Defto A: If ( Λ, Ω, where ( λ λ λ whch the probablty dstrbutos,,..., Defto A. uppose that ( Λ,,..., s a expermet type, the σ-algebra o λ λ λ are defed s deoted by ( (,,...,, σ Ω.
More informationLebesgue Measure of Generalized Cantor Set
Aals of Pure ad Appled Mathematcs Vol., No.,, -8 ISSN: -8X P), -888ole) Publshed o 8 May www.researchmathsc.org Aals of Lebesgue Measure of Geeralzed ator Set Md. Jahurul Islam ad Md. Shahdul Islam Departmet
More informationDerivation of 3-Point Block Method Formula for Solving First Order Stiff Ordinary Differential Equations
Dervato of -Pot Block Method Formula for Solvg Frst Order Stff Ordary Dfferetal Equatos Kharul Hamd Kharul Auar, Kharl Iskadar Othma, Zara Bb Ibrahm Abstract Dervato of pot block method formula wth costat
More informationPart 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971))
art 4b Asymptotc Results for MRR usg RESS Recall that the RESS statstc s a specal type of cross valdato procedure (see Alle (97)) partcular to the regresso problem ad volves fdg Y $,, the estmate at the
More informationComparing Different Estimators of three Parameters for Transmuted Weibull Distribution
Global Joural of Pure ad Appled Mathematcs. ISSN 0973-768 Volume 3, Number 9 (207), pp. 55-528 Research Ida Publcatos http://www.rpublcato.com Comparg Dfferet Estmators of three Parameters for Trasmuted
More informationRegression and the LMS Algorithm
CSE 556: Itroducto to Neural Netorks Regresso ad the LMS Algorthm CSE 556: Regresso 1 Problem statemet CSE 556: Regresso Lear regresso th oe varable Gve a set of N pars of data {, d }, appromate d b a
More informationA tighter lower bound on the circuit size of the hardest Boolean functions
Electroc Colloquum o Computatoal Complexty, Report No. 86 2011) A tghter lower boud o the crcut sze of the hardest Boolea fuctos Masak Yamamoto Abstract I [IPL2005], Fradse ad Mlterse mproved bouds o the
More informationIntroduction to local (nonparametric) density estimation. methods
Itroducto to local (oparametrc) desty estmato methods A slecture by Yu Lu for ECE 66 Sprg 014 1. Itroducto Ths slecture troduces two local desty estmato methods whch are Parze desty estmato ad k-earest
More information7.0 Equality Contraints: Lagrange Multipliers
Systes Optzato 7.0 Equalty Cotrats: Lagrage Multplers Cosder the zato of a o-lear fucto subject to equalty costrats: g f() R ( ) 0 ( ) (7.) where the g ( ) are possbly also olear fuctos, ad < otherwse
More informationSimulation Output Analysis
Smulato Output Aalyss Summary Examples Parameter Estmato Sample Mea ad Varace Pot ad Iterval Estmato ermatg ad o-ermatg Smulato Mea Square Errors Example: Sgle Server Queueg System x(t) S 4 S 4 S 3 S 5
More informationBlock-Based Compact Thermal Modeling of Semiconductor Integrated Circuits
Block-Based Compact hermal Modelg of Semcoductor Itegrated Crcuts Master s hess Defese Caddate: Jg Ba Commttee Members: Dr. Mg-Cheg Cheg Dr. Daqg Hou Dr. Robert Schllg July 27, 2009 Outle Itroducto Backgroud
More informationStrong Convergence of Weighted Averaged Approximants of Asymptotically Nonexpansive Mappings in Banach Spaces without Uniform Convexity
BULLETIN of the MALAYSIAN MATHEMATICAL SCIENCES SOCIETY Bull. Malays. Math. Sc. Soc. () 7 (004), 5 35 Strog Covergece of Weghted Averaged Appromats of Asymptotcally Noepasve Mappgs Baach Spaces wthout
More informationOn Modified Interval Symmetric Single-Step Procedure ISS2-5D for the Simultaneous Inclusion of Polynomial Zeros
It. Joural of Math. Aalyss, Vol. 7, 2013, o. 20, 983-988 HIKARI Ltd, www.m-hkar.com O Modfed Iterval Symmetrc Sgle-Step Procedure ISS2-5D for the Smultaeous Icluso of Polyomal Zeros 1 Nora Jamalud, 1 Masor
More informationTESTS BASED ON MAXIMUM LIKELIHOOD
ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal
More informationResearch Article A New Iterative Method for Common Fixed Points of a Finite Family of Nonexpansive Mappings
Hdaw Publshg Corporato Iteratoal Joural of Mathematcs ad Mathematcal Sceces Volume 009, Artcle ID 391839, 9 pages do:10.1155/009/391839 Research Artcle A New Iteratve Method for Commo Fxed Pots of a Fte
More informationBinary classification: Support Vector Machines
CS 57 Itroducto to AI Lecture 6 Bar classfcato: Support Vector Maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Supervsed learg Data: D { D, D,.., D} a set of eamples D, (,,,,,
More informationSolution of General Dual Fuzzy Linear Systems. Using ABS Algorithm
Appled Mathematcal Sceces, Vol 6, 0, o 4, 63-7 Soluto of Geeral Dual Fuzzy Lear Systems Usg ABS Algorthm M A Farborz Aragh * ad M M ossezadeh Departmet of Mathematcs, Islamc Azad Uversty Cetral ehra Brach,
More information2006 Jamie Trahan, Autar Kaw, Kevin Martin University of South Florida United States of America
SOLUTION OF SYSTEMS OF SIMULTANEOUS LINEAR EQUATIONS Gauss-Sedel Method 006 Jame Traha, Autar Kaw, Kev Mart Uversty of South Florda Uted States of Amerca kaw@eg.usf.edu Itroducto Ths worksheet demostrates
More informationA Method for Damping Estimation Based On Least Square Fit
Amerca Joural of Egeerg Research (AJER) 5 Amerca Joural of Egeerg Research (AJER) e-issn: 3-847 p-issn : 3-936 Volume-4, Issue-7, pp-5-9 www.ajer.org Research Paper Ope Access A Method for Dampg Estmato
More information9 U-STATISTICS. Eh =(m!) 1 Eh(X (1),..., X (m ) ) i.i.d
9 U-STATISTICS Suppose,,..., are P P..d. wth CDF F. Our goal s to estmate the expectato t (P)=Eh(,,..., m ). Note that ths expectato requres more tha oe cotrast to E, E, or Eh( ). Oe example s E or P((,
More informationMaximum Likelihood Estimation
Marquette Uverst Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 08 b Marquette Uverst Maxmum Lkelhood Estmato We have bee sag that ~
More informationmeans the first term, a2 means the term, etc. Infinite Sequences: follow the same pattern forever.
9.4 Sequeces ad Seres Pre Calculus 9.4 SEQUENCES AND SERIES Learg Targets:. Wrte the terms of a explctly defed sequece.. Wrte the terms of a recursvely defed sequece. 3. Determe whether a sequece s arthmetc,
More informationL5 Polynomial / Spline Curves
L5 Polyomal / Sple Curves Cotets Coc sectos Polyomal Curves Hermte Curves Bezer Curves B-Sples No-Uform Ratoal B-Sples (NURBS) Mapulato ad Represetato of Curves Types of Curve Equatos Implct: Descrbe a
More information18.413: Error Correcting Codes Lab March 2, Lecture 8
18.413: Error Correctg Codes Lab March 2, 2004 Lecturer: Dael A. Spelma Lecture 8 8.1 Vector Spaces A set C {0, 1} s a vector space f for x all C ad y C, x + y C, where we take addto to be compoet wse
More informationThe internal structure of natural numbers, one method for the definition of large prime numbers, and a factorization test
Fal verso The teral structure of atural umbers oe method for the defto of large prme umbers ad a factorzato test Emmaul Maousos APM Isttute for the Advacemet of Physcs ad Mathematcs 3 Poulou str. 53 Athes
More informationA Remark on the Uniform Convergence of Some Sequences of Functions
Advaces Pure Mathematcs 05 5 57-533 Publshed Ole July 05 ScRes. http://www.scrp.org/joural/apm http://dx.do.org/0.436/apm.05.59048 A Remark o the Uform Covergece of Some Sequeces of Fuctos Guy Degla Isttut
More informationAn Indian Journal FULL PAPER ABSTRACT KEYWORDS. Trade Science Inc. Research on scheme evaluation method of automation mechatronic systems
[ype text] [ype text] [ype text] ISSN : 0974-7435 Volume 0 Issue 6 Boechology 204 Ida Joural FULL PPER BIJ, 0(6, 204 [927-9275] Research o scheme evaluato method of automato mechatroc systems BSRC Che
More informationDKA method for single variable holomorphic functions
DKA method for sgle varable holomorphc fuctos TOSHIAKI ITOH Itegrated Arts ad Natural Sceces The Uversty of Toushma -, Mamhosama, Toushma, 770-8502 JAPAN Abstract: - Durad-Kerer-Aberth (DKA method for
More informationρ < 1 be five real numbers. The
Lecture o BST 63: Statstcal Theory I Ku Zhag, /0/006 Revew for the prevous lecture Deftos: covarace, correlato Examples: How to calculate covarace ad correlato Theorems: propertes of correlato ad covarace
More informationA New Method for Decision Making Based on Soft Matrix Theory
Joural of Scetfc esearch & eports 3(5): 0-7, 04; rtcle o. JS.04.5.00 SCIENCEDOMIN teratoal www.scecedoma.org New Method for Decso Mag Based o Soft Matrx Theory Zhmg Zhag * College of Mathematcs ad Computer
More informationMOLECULAR VIBRATIONS
MOLECULAR VIBRATIONS Here we wsh to vestgate molecular vbratos ad draw a smlarty betwee the theory of molecular vbratos ad Hückel theory. 1. Smple Harmoc Oscllator Recall that the eergy of a oe-dmesoal
More informationF. Inequalities. HKAL Pure Mathematics. 進佳數學團隊 Dr. Herbert Lam 林康榮博士. [Solution] Example Basic properties
進佳數學團隊 Dr. Herbert Lam 林康榮博士 HKAL Pure Mathematcs F. Ieualtes. Basc propertes Theorem Let a, b, c be real umbers. () If a b ad b c, the a c. () If a b ad c 0, the ac bc, but f a b ad c 0, the ac bc. Theorem
More informationTaylor s Series and Interpolation. Interpolation & Curve-fitting. CIS Interpolation. Basic Scenario. Taylor Series interpolates at a specific
CIS 54 - Iterpolato Roger Crawfs Basc Scearo We are able to prod some fucto, but do ot kow what t really s. Ths gves us a lst of data pots: [x,f ] f(x) f f + x x + August 2, 25 OSU/CIS 54 3 Taylor s Seres
More informationMultivariate Transformation of Variables and Maximum Likelihood Estimation
Marquette Uversty Multvarate Trasformato of Varables ad Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Assocate Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Copyrght 03 by Marquette Uversty
More informationLecture 9: Tolerant Testing
Lecture 9: Tolerat Testg Dael Kae Scrbe: Sakeerth Rao Aprl 4, 07 Abstract I ths lecture we prove a quas lear lower boud o the umber of samples eeded to do tolerat testg for L dstace. Tolerat Testg We have
More informationLecture 7. Confidence Intervals and Hypothesis Tests in the Simple CLR Model
Lecture 7. Cofdece Itervals ad Hypothess Tests the Smple CLR Model I lecture 6 we troduced the Classcal Lear Regresso (CLR) model that s the radom expermet of whch the data Y,,, K, are the outcomes. The
More informationarxiv: v1 [cs.lg] 22 Feb 2015
SDCA wthout Dualty Sha Shalev-Shwartz arxv:50.0677v cs.lg Feb 05 Abstract Stochastc Dual Coordate Ascet s a popular method for solvg regularzed loss mmzato for the case of covex losses. I ths paper we
More informationMATH 247/Winter Notes on the adjoint and on normal operators.
MATH 47/Wter 00 Notes o the adjot ad o ormal operators I these otes, V s a fte dmesoal er product space over, wth gve er * product uv, T, S, T, are lear operators o V U, W are subspaces of V Whe we say
More informationConvergence of the Desroziers scheme and its relation to the lag innovation diagnostic
Covergece of the Desrozers scheme ad ts relato to the lag ovato dagostc chard Méard Evromet Caada, Ar Qualty esearch Dvso World Weather Ope Scece Coferece Motreal, August 9, 04 o t t O x x x y x y Oservato
More informationChapter 9 Jordan Block Matrices
Chapter 9 Jorda Block atrces I ths chapter we wll solve the followg problem. Gve a lear operator T fd a bass R of F such that the matrx R (T) s as smple as possble. f course smple s a matter of taste.
More information4 Inner Product Spaces
11.MH1 LINEAR ALGEBRA Summary Notes 4 Ier Product Spaces Ier product s the abstracto to geeral vector spaces of the famlar dea of the scalar product of two vectors or 3. I what follows, keep these key
More informationChapter 4 (Part 1): Non-Parametric Classification (Sections ) Pattern Classification 4.3) Announcements
Aoucemets No-Parametrc Desty Estmato Techques HW assged Most of ths lecture was o the blacboard. These sldes cover the same materal as preseted DHS Bometrcs CSE 90-a Lecture 7 CSE90a Fall 06 CSE90a Fall
More information5 Short Proofs of Simplified Stirling s Approximation
5 Short Proofs of Smplfed Strlg s Approxmato Ofr Gorodetsky, drtymaths.wordpress.com Jue, 20 0 Itroducto Strlg s approxmato s the followg (somewhat surprsg) approxmato of the factoral,, usg elemetary fuctos:
More informationChapter 4 Multiple Random Variables
Revew for the prevous lecture: Theorems ad Examples: How to obta the pmf (pdf) of U = g (, Y) ad V = g (, Y) Chapter 4 Multple Radom Varables Chapter 44 Herarchcal Models ad Mxture Dstrbutos Examples:
More informationA Markov Chain Competition Model
Academc Forum 3 5-6 A Marov Cha Competto Model Mchael Lloyd, Ph.D. Mathematcs ad Computer Scece Abstract A brth ad death cha for two or more speces s examed aalytcally ad umercally. Descrpto of the Model
More informationarxiv:math/ v1 [math.gm] 8 Dec 2005
arxv:math/05272v [math.gm] 8 Dec 2005 A GENERALIZATION OF AN INEQUALITY FROM IMO 2005 NIKOLAI NIKOLOV The preset paper was spred by the thrd problem from the IMO 2005. A specal award was gve to Yure Boreko
More information= lim. (x 1 x 2... x n ) 1 n. = log. x i. = M, n
.. Soluto of Problem. M s obvously cotuous o ], [ ad ], [. Observe that M x,..., x ) M x,..., x ) )..) We ext show that M s odecreasg o ], [. Of course.) mles that M s odecreasg o ], [ as well. To show
More informationTransforms that are commonly used are separable
Trasforms s Trasforms that are commoly used are separable Eamples: Two-dmesoal DFT DCT DST adamard We ca the use -D trasforms computg the D separable trasforms: Take -D trasform of the rows > rows ( )
More informationSTRONG CONSISTENCY FOR SIMPLE LINEAR EV MODEL WITH v/ -MIXING
Joural of tatstcs: Advaces Theory ad Alcatos Volume 5, Number, 6, Pages 3- Avalable at htt://scetfcadvaces.co. DOI: htt://d.do.org/.864/jsata_7678 TRONG CONITENCY FOR IMPLE LINEAR EV MODEL WITH v/ -MIXING
More informationBeam Warming Second-Order Upwind Method
Beam Warmg Secod-Order Upwd Method Petr Valeta Jauary 6, 015 Ths documet s a part of the assessmet work for the subject 1DRP Dfferetal Equatos o Computer lectured o FNSPE CTU Prague. Abstract Ths documet
More informationTHE ROYAL STATISTICAL SOCIETY 2009 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA MODULAR FORMAT MODULE 2 STATISTICAL INFERENCE
THE ROYAL STATISTICAL SOCIETY 009 EXAMINATIONS SOLUTIONS GRADUATE DIPLOMA MODULAR FORMAT MODULE STATISTICAL INFERENCE The Socety provdes these solutos to assst caddates preparg for the examatos future
More informationEVALUATION OF FUNCTIONAL INTEGRALS BY MEANS OF A SERIES AND THE METHOD OF BOREL TRANSFORM
EVALUATION OF FUNCTIONAL INTEGRALS BY MEANS OF A SERIES AND THE METHOD OF BOREL TRANSFORM Jose Javer Garca Moreta Ph. D research studet at the UPV/EHU (Uversty of Basque coutry) Departmet of Theoretcal
More informationSimple Linear Regression
Statstcal Methods I (EST 75) Page 139 Smple Lear Regresso Smple regresso applcatos are used to ft a model descrbg a lear relatoshp betwee two varables. The aspects of least squares regresso ad correlato
More informationChapter 14 Logistic Regression Models
Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as
More information02/15/04 INTERESTING FINITE AND INFINITE PRODUCTS FROM SIMPLE ALGEBRAIC IDENTITIES
0/5/04 ITERESTIG FIITE AD IFIITE PRODUCTS FROM SIMPLE ALGEBRAIC IDETITIES Thomas J Osler Mathematcs Departmet Rowa Uversty Glassboro J 0808 Osler@rowaedu Itroducto The dfferece of two squares, y = + y
More informationA Collocation Method for Solving Abel s Integral Equations of First and Second Kinds
A Collocato Method for Solvg Abel s Itegral Equatos of Frst ad Secod Kds Abbas Saadatmad a ad Mehd Dehgha b a Departmet of Mathematcs, Uversty of Kasha, Kasha, Ira b Departmet of Appled Mathematcs, Faculty
More informationMu Sequences/Series Solutions National Convention 2014
Mu Sequeces/Seres Solutos Natoal Coveto 04 C 6 E A 6C A 6 B B 7 A D 7 D C 7 A B 8 A B 8 A C 8 E 4 B 9 B 4 E 9 B 4 C 9 E C 0 A A 0 D B 0 C C Usg basc propertes of arthmetc sequeces, we fd a ad bm m We eed
More informationX X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then
Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers
More informationEstimation of Stress- Strength Reliability model using finite mixture of exponential distributions
Iteratoal Joural of Computatoal Egeerg Research Vol, 0 Issue, Estmato of Stress- Stregth Relablty model usg fte mxture of expoetal dstrbutos K.Sadhya, T.S.Umamaheswar Departmet of Mathematcs, Lal Bhadur
More informationAnalysis of Variance with Weibull Data
Aalyss of Varace wth Webull Data Lahaa Watthaacheewaul Abstract I statstcal data aalyss by aalyss of varace, the usual basc assumptos are that the model s addtve ad the errors are radomly, depedetly, ad
More information