LECTURE 24 LECTURE OUTLINE

Size: px
Start display at page:

Download "LECTURE 24 LECTURE OUTLINE"

Transcription

1 LECTURE 24 LECTURE OUTLINE Gradet proxmal mmzato method Noquadratc proxmal algorthms Etropy mmzato algorthm Expoetal augmeted Lagraga mehod Etropc descet algorthm ************************************** Refereces: Bec, A., ad Teboulle, M., 200. Gradet- Based Algorthms wth Applcatos to Sgal Recovery Problems, Covex Optmzato Sgal Processg ad Commucatos (Y. Eldar ad D. Palomar, eds.), Cambrdge Uversty Press, pp Bec, A., ad Teboulle, M., Mrror Descet ad Nolear Projected Subgradet Methods for Covex Optmzato, Operatos Research Letters, Vol. 3, pp Bertseas, D. P., 999. Nolear Programmg, Athea Scetfc, Belmot, MA. All fgures are courtesy of Athea Scetfc, ad are used wth permsso.

2 PROXIMAL AND GRADIENT PROJECTION Proxmal algorthm to mmze covex f over closed covex X { x + arg m f(x)+ x x 2 x X 2c f(x ) γ f(x) γ 2c x x 2 x x + x x Let f be dfferetable ad assume f(x) f(y) L x y, x, y X Defe the lear approxmato fucto at x!(y; x) =f(x)+ f(x) (y x), y Coecto of proxmal wth gradet projecto { ( y = arg m!(z; x)+ z X 2α z x 2 = P X x α f(x) 2

3 GRADIENT-PROXIMAL METHOD I Mmze f(x)+g(x) over x X, where X: closed covex, f, g: covex, f s dfferetable. Gradet-proxmal method: { x + arg m!(x; x )+g(x)+ x X 2α x x 2 Recall ey equalty: For all x, y X L f(y)!(y; x)+ y x 2 2 Cost reducto for α /L: L f(x + )+g(x + )!(x + ; x )+ x 2 + x 2 + g(x + )!(x + ; x )+g(x + )+ x + x 2 2α!(x ; x )+g(x ) = f(x )+g(x ) Ths s a ey sght for the covergece aalyss. 3

4 GRADIENT-PROXIMAL METHOD II Equvalet defto of gradet-proxmal: x + arg m x X z = x α f(x ) { g(x)+ 2α x z 2 Smplfes the mplemetato of proxmal, by usg gradet terato to deal wth the case of a coveet compoet f Ths s smlar to cremetal subgradet-proxmal method, but the gradet-proxmal method does ot exted to the case where the cost cossts of the sum of multple compoets. Allows a costat stepsze (uder the restrcto α /L). Ths does ot exted to cremetal methods. Le all gradet ad subgradet methods, covergece ca be slow. There are specal cases where the method ca be frutfully appled (see the referece by Bec ad Teboulle). 4

5 GENERALIZED PROXIMAL ALGORITHM Itroduce a geeral regularzato term D : x + arg m x X Example: Bregma f(x)+d (x, x ) dstace fucto D (x, y) = ( (x) ( y) (y) (x c y), where : (, ] s a covex fucto, dfferetable wth a ope set cotag dom(f), ad c s a postve pealty parameter. All the deas for applcatos ad coectos of the quadratc form of the proxmal algorthm exted to the oquadratc case (although the aalyss may ot be trval). I partcular we have: A dual proxmal algorthm (based o Fechel dualty) Equvalece wth (oquadratc) augmeted Lagragea method Combatos wth polyhedral approxmatos (budle-type methods) Icremetal subgradet-proxmal methods Nolear gradet projecto algorthms 5

6 ENTROPY MINIMIZATION ALGORITHM A specal case volvg etropy regularzato: x x + arg m f(x)+ x l x X c x = where x 0 ad all subsequet x have postve compoets We use Fechel dualty to obta a dual form of ths mmzato Note: The logarthmc fucto p(x) = x(l x ) ad the expoetal fucto are a cojugate par. The dual problem s y + arg m f x>0, 0 f x =0, f x<0, p (y) =e y f (y)+ x e c y y c = 6

7 EXPONENTIAL AUGMENTED LAGRANGIAN The dual proxmal terato s x = x e c y + +, =,..., where y + s obtaed from the dual proxmal: y + arg m y f (y)+ c = x e c y A specal case for the covex problem mmze f(x) subject to g (x) 0,...,g r (x) 0, x X s the expoetal augmeted Lagragea method Cossts of ucostraed mmzatos r x arg m f(x)+ µ j e c g j (x), x X c j= followed by the multpler teratos j µ j j c g j (x ) + = µ e, j =,...,r 7

8 NONLINEAR PROJECTION ALGORITHM Subgradet projecto wth geeral regularzato term D : x + arg m f(x )+ f(x ) (x x )+D (x, x ) x X where f(x ) s a subgradet of f at x. called mrror descet method. Also Learzato of f smplfes the mmzato The use of oquadratc learzato s useful problems wth specal structure Etropc descet method: Mmze f(x) over the ut smplex X = x 0 x =. Method: x + arg m = x g + x X α = where g are the compoets of f(x ). l x x Ths mmzato ca be doe closed form: x + = x e α g, j =,..., x j e α g j= 8

9 MIT OpeCourseWare Covex Aalyss ad Optmzato Sprg 202 For formato about ctg these materals or our Terms of Use, vst:

arxiv: v1 [cs.lg] 22 Feb 2015

arxiv: v1 [cs.lg] 22 Feb 2015 SDCA wthout Dualty Sha Shalev-Shwartz arxv:50.0677v cs.lg Feb 05 Abstract Stochastc Dual Coordate Ascet s a popular method for solvg regularzed loss mmzato for the case of covex losses. I ths paper we

More information

Parallel Multi-splitting Proximal Method for Star Networks

Parallel Multi-splitting Proximal Method for Star Networks Parallel Mult-splttg Proxmal Method for Star Networks Erm We Departmet of Electrcal Egeerg ad Computer Scece Northwester Uversty Evasto, IL 600 erm.we@orthwester.edu Abstract We develop a parallel algorthm

More information

Support vector machines

Support vector machines CS 75 Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Outle Outle: Algorthms for lear decso boudary Support vector maches Mamum marg hyperplae.

More information

ECE 421/599 Electric Energy Systems 7 Optimal Dispatch of Generation. Instructor: Kai Sun Fall 2014

ECE 421/599 Electric Energy Systems 7 Optimal Dispatch of Generation. Instructor: Kai Sun Fall 2014 ECE 4/599 Electrc Eergy Systems 7 Optmal Dspatch of Geerato Istructor: Ka Su Fall 04 Backgroud I a practcal power system, the costs of geeratg ad delverg electrcty from power plats are dfferet (due to

More information

A new type of optimization method based on conjugate directions

A new type of optimization method based on conjugate directions A ew type of optmzato method based o cojugate drectos Pa X Scece School aj Uversty of echology ad Educato (UE aj Cha e-mal: pax94@sacom Abstract A ew type of optmzato method based o cojugate drectos s

More information

Supplementary material for the paper Optimal Transport for Domain Adaptation

Supplementary material for the paper Optimal Transport for Domain Adaptation 1 Supplemetary materal for the paper Optmal Trasport for Doma Adaptato Ncolas Courty, Member, IEEE, Rem Flamary, Devs Tua, Seor Member, IEEE, Ala Rakotomamojy, Member, IEEE I. OPTIMAL TRANSPORT FOR AFFINE

More information

Support vector machines II

Support vector machines II CS 75 Mache Learg Lecture Support vector maches II Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Learl separable classes Learl separable classes: here s a hperplae that separates trag staces th o error

More information

Functions of Random Variables

Functions of Random Variables Fuctos of Radom Varables Chapter Fve Fuctos of Radom Varables 5. Itroducto A geeral egeerg aalyss model s show Fg. 5.. The model output (respose) cotas the performaces of a system or product, such as weght,

More information

An Introduction to. Support Vector Machine

An Introduction to. Support Vector Machine A Itroducto to Support Vector Mache Support Vector Mache (SVM) A classfer derved from statstcal learg theory by Vapk, et al. 99 SVM became famous whe, usg mages as put, t gave accuracy comparable to eural-etwork

More information

COMPROMISE HYPERSPHERE FOR STOCHASTIC DOMINANCE MODEL

COMPROMISE HYPERSPHERE FOR STOCHASTIC DOMINANCE MODEL Sebasta Starz COMPROMISE HYPERSPHERE FOR STOCHASTIC DOMINANCE MODEL Abstract The am of the work s to preset a method of rakg a fte set of dscrete radom varables. The proposed method s based o two approaches:

More information

CS 1675 Introduction to Machine Learning Lecture 12 Support vector machines

CS 1675 Introduction to Machine Learning Lecture 12 Support vector machines CS 675 Itroducto to Mache Learg Lecture Support vector maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square Mdterm eam October 9, 7 I-class eam Closed book Stud materal: Lecture otes Correspodg chapters

More information

Supervised learning: Linear regression Logistic regression

Supervised learning: Linear regression Logistic regression CS 57 Itroducto to AI Lecture 4 Supervsed learg: Lear regresso Logstc regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Data: D { D D.. D D Supervsed learg d a set of eamples s

More information

BERNSTEIN COLLOCATION METHOD FOR SOLVING NONLINEAR DIFFERENTIAL EQUATIONS. Aysegul Akyuz Dascioglu and Nese Isler

BERNSTEIN COLLOCATION METHOD FOR SOLVING NONLINEAR DIFFERENTIAL EQUATIONS. Aysegul Akyuz Dascioglu and Nese Isler Mathematcal ad Computatoal Applcatos, Vol. 8, No. 3, pp. 293-300, 203 BERNSTEIN COLLOCATION METHOD FOR SOLVING NONLINEAR DIFFERENTIAL EQUATIONS Aysegul Ayuz Dascoglu ad Nese Isler Departmet of Mathematcs,

More information

Chapter 4 Multiple Random Variables

Chapter 4 Multiple Random Variables Revew for the prevous lecture: Theorems ad Examples: How to obta the pmf (pdf) of U = g (, Y) ad V = g (, Y) Chapter 4 Multple Radom Varables Chapter 44 Herarchcal Models ad Mxture Dstrbutos Examples:

More information

An Accelerated Proximal Coordinate Gradient Method

An Accelerated Proximal Coordinate Gradient Method A Accelerated Proxmal Coordate Gradet Method Qhag L Uversty of Iowa Iowa Cty IA USA qhag-l@uowaedu Zhaosog Lu Smo Fraser Uversty Buraby BC Caada zhaosog@sfuca L Xao Mcrosoft Research Redmod WA USA lxao@mcrosoftcom

More information

X ε ) = 0, or equivalently, lim

X ε ) = 0, or equivalently, lim Revew for the prevous lecture Cocepts: order statstcs Theorems: Dstrbutos of order statstcs Examples: How to get the dstrbuto of order statstcs Chapter 5 Propertes of a Radom Sample Secto 55 Covergece

More information

CSE 5526: Introduction to Neural Networks Linear Regression

CSE 5526: Introduction to Neural Networks Linear Regression CSE 556: Itroducto to Neural Netorks Lear Regresso Part II 1 Problem statemet Part II Problem statemet Part II 3 Lear regresso th oe varable Gve a set of N pars of data , appromate d by a lear fucto

More information

3. Basic Concepts: Consequences and Properties

3. Basic Concepts: Consequences and Properties : 3. Basc Cocepts: Cosequeces ad Propertes Markku Jutt Overvew More advaced cosequeces ad propertes of the basc cocepts troduced the prevous lecture are derved. Source The materal s maly based o Sectos.6.8

More information

CS 2750 Machine Learning. Lecture 8. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x

CS 2750 Machine Learning. Lecture 8. Linear regression. CS 2750 Machine Learning. Linear regression. is a linear combination of input components x CS 75 Mache Learg Lecture 8 Lear regresso Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 75 Mache Learg Lear regresso Fucto f : X Y s a lear combato of put compoets f + + + K d d K k - parameters

More information

Lecture 16: Backpropogation Algorithm Neural Networks with smooth activation functions

Lecture 16: Backpropogation Algorithm Neural Networks with smooth activation functions CO-511: Learg Theory prg 2017 Lecturer: Ro Lv Lecture 16: Bacpropogato Algorthm Dsclamer: These otes have ot bee subected to the usual scruty reserved for formal publcatos. They may be dstrbuted outsde

More information

A conic cutting surface method for linear-quadraticsemidefinite

A conic cutting surface method for linear-quadraticsemidefinite A coc cuttg surface method for lear-quadratcsemdefte programmg Mohammad R. Osoorouch Calfora State Uversty Sa Marcos Sa Marcos, CA Jot wor wth Joh E. Mtchell RPI July 3, 2008 Outle: Secod-order coe: defto

More information

Entropies & Information Theory

Entropies & Information Theory Etropes & Iformato Theory LECTURE II Nlajaa Datta Uversty of Cambrdge,U.K. See lecture otes o: http://www.q.damtp.cam.ac.uk/ode/223 quatum system States (of a physcal system): Hlbert space (fte-dmesoal)

More information

CHAPTER 4 RADICAL EXPRESSIONS

CHAPTER 4 RADICAL EXPRESSIONS 6 CHAPTER RADICAL EXPRESSIONS. The th Root of a Real Number A real umber a s called the th root of a real umber b f Thus, for example: s a square root of sce. s also a square root of sce ( ). s a cube

More information

Research Article A New Iterative Method for Common Fixed Points of a Finite Family of Nonexpansive Mappings

Research Article A New Iterative Method for Common Fixed Points of a Finite Family of Nonexpansive Mappings Hdaw Publshg Corporato Iteratoal Joural of Mathematcs ad Mathematcal Sceces Volume 009, Artcle ID 391839, 9 pages do:10.1155/009/391839 Research Artcle A New Iteratve Method for Commo Fxed Pots of a Fte

More information

Analysis of Lagrange Interpolation Formula

Analysis of Lagrange Interpolation Formula P IJISET - Iteratoal Joural of Iovatve Scece, Egeerg & Techology, Vol. Issue, December 4. www.jset.com ISS 348 7968 Aalyss of Lagrage Iterpolato Formula Vjay Dahya PDepartmet of MathematcsMaharaja Surajmal

More information

PROJECTION PROBLEM FOR REGULAR POLYGONS

PROJECTION PROBLEM FOR REGULAR POLYGONS Joural of Mathematcal Sceces: Advaces ad Applcatos Volume, Number, 008, Pages 95-50 PROJECTION PROBLEM FOR REGULAR POLYGONS College of Scece Bejg Forestry Uversty Bejg 0008 P. R. Cha e-mal: sl@bjfu.edu.c

More information

A Robust Total Least Mean Square Algorithm For Nonlinear Adaptive Filter

A Robust Total Least Mean Square Algorithm For Nonlinear Adaptive Filter A Robust otal east Mea Square Algorthm For Nolear Adaptve Flter Ruxua We School of Electroc ad Iformato Egeerg X'a Jaotog Uversty X'a 70049, P.R. Cha rxwe@chare.com Chogzhao Ha, azhe u School of Electroc

More information

Variable Metric Forward-Backward Splitting with Applications to Monotone Inclusions in Duality

Variable Metric Forward-Backward Splitting with Applications to Monotone Inclusions in Duality Varable Metrc Forward-Backward Splttg wth Applcatos to Mootoe Iclusos Dualty Patrck L. Combettes ad B`ăg Côg Vũ UPMC Uversté Pars 06 Laboratore Jacques-Lous Los UMR CNRS 7598 75005 Pars, Frace plc@math.jusseu.fr,

More information

A COMPARATIVE STUDY OF THE METHODS OF SOLVING NON-LINEAR PROGRAMMING PROBLEM

A COMPARATIVE STUDY OF THE METHODS OF SOLVING NON-LINEAR PROGRAMMING PROBLEM DAODIL INTERNATIONAL UNIVERSITY JOURNAL O SCIENCE AND TECHNOLOGY, VOLUME, ISSUE, JANUARY 9 A COMPARATIVE STUDY O THE METHODS O SOLVING NON-LINEAR PROGRAMMING PROBLEM Bmal Chadra Das Departmet of Tetle

More information

Binary classification: Support Vector Machines

Binary classification: Support Vector Machines CS 57 Itroducto to AI Lecture 6 Bar classfcato: Support Vector Maches Mlos Hauskrecht mlos@cs.ptt.edu 539 Seott Square CS 57 Itro to AI Supervsed learg Data: D { D, D,.., D} a set of eamples D, (,,,,,

More information

NP!= P. By Liu Ran. Table of Contents. The P versus NP problem is a major unsolved problem in computer

NP!= P. By Liu Ran. Table of Contents. The P versus NP problem is a major unsolved problem in computer NP!= P By Lu Ra Table of Cotets. Itroduce 2. Prelmary theorem 3. Proof 4. Expla 5. Cocluso. Itroduce The P versus NP problem s a major usolved problem computer scece. Iformally, t asks whether a computer

More information

Nonlinear Blind Source Separation Using Hybrid Neural Networks*

Nonlinear Blind Source Separation Using Hybrid Neural Networks* Nolear Bld Source Separato Usg Hybrd Neural Networks* Chu-Hou Zheg,2, Zh-Ka Huag,2, chael R. Lyu 3, ad Tat-g Lok 4 Itellget Computg Lab, Isttute of Itellget aches, Chese Academy of Sceces, P.O.Box 3, Hefe,

More information

0/1 INTEGER PROGRAMMING AND SEMIDEFINTE PROGRAMMING

0/1 INTEGER PROGRAMMING AND SEMIDEFINTE PROGRAMMING CONVEX OPIMIZAION AND INERIOR POIN MEHODS FINAL PROJEC / INEGER PROGRAMMING AND SEMIDEFINE PROGRAMMING b Luca Buch ad Natala Vktorova CONENS:.Itroducto.Formulato.Applcato to Kapsack Problem 4.Cuttg Plaes

More information

1 Convergence of the Arnoldi method for eigenvalue problems

1 Convergence of the Arnoldi method for eigenvalue problems Lecture otes umercal lear algebra Arold method covergece Covergece of the Arold method for egevalue problems Recall that, uless t breaks dow, k steps of the Arold method geerates a orthogoal bass of a

More information

Wavelet Basics. (A Beginner s Introduction) J. S. Marron Department of Statistics University of North Carolina

Wavelet Basics. (A Beginner s Introduction) J. S. Marron Department of Statistics University of North Carolina Wavelet Bascs (A Beger s Itroducto) J. S. Marro Departmet o Statstcs Uversty o North Carola Some reereces: Marro, J. S. (999) Spectral vew o wavelets ad olear regresso, Bayesa Ierece Wavelet-Based Models,

More information

Stochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimization

Stochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimization Stochastc Prmal-Dual Coordate Method for Regularzed Emprcal Rsk Mmzato Yuche Zhag L Xao September 24 Abstract We cosder a geerc covex optmzato problem assocated wth regularzed emprcal rsk mmzato of lear

More information

CONSTRUCTING GENERALIZED MEAN FUNCTIONS USING CONVEX FUNCTIONS WITH REGULARITY CONDITIONS

CONSTRUCTING GENERALIZED MEAN FUNCTIONS USING CONVEX FUNCTIONS WITH REGULARITY CONDITIONS CONSTRUCTING GENERALIZED MEAN FUNCTIONS USING CONVEX FUNCTIONS WITH REGULARITY CONDITIONS YUN-BIN ZHAO, SHU-CHERNG FANG, AND DUAN LI Abstract. The geeralzed mea fucto has bee wdely used covex aalyss ad

More information

Feature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture)

Feature Selection: Part 2. 1 Greedy Algorithms (continued from the last lecture) CSE 546: Mache Learg Lecture 6 Feature Selecto: Part 2 Istructor: Sham Kakade Greedy Algorthms (cotued from the last lecture) There are varety of greedy algorthms ad umerous amg covetos for these algorthms.

More information

PTAS for Bin-Packing

PTAS for Bin-Packing CS 663: Patter Matchg Algorthms Scrbe: Che Jag /9/00. Itroducto PTAS for B-Packg The B-Packg problem s NP-hard. If we use approxmato algorthms, the B-Packg problem could be solved polyomal tme. For example,

More information

n -dimensional vectors follow naturally from the one

n -dimensional vectors follow naturally from the one B. Vectors ad sets B. Vectors Ecoomsts study ecoomc pheomea by buldg hghly stylzed models. Uderstadg ad makg use of almost all such models requres a hgh comfort level wth some key mathematcal sklls. I

More information

Regression and the LMS Algorithm

Regression and the LMS Algorithm CSE 556: Itroducto to Neural Netorks Regresso ad the LMS Algorthm CSE 556: Regresso 1 Problem statemet CSE 556: Regresso Lear regresso th oe varable Gve a set of N pars of data {, d }, appromate d b a

More information

( ) 2 2. Multi-Layer Refraction Problem Rafael Espericueta, Bakersfield College, November, 2006

( ) 2 2. Multi-Layer Refraction Problem Rafael Espericueta, Bakersfield College, November, 2006 Mult-Layer Refracto Problem Rafael Espercueta, Bakersfeld College, November, 006 Lght travels at dfferet speeds through dfferet meda, but refracts at layer boudares order to traverse the least-tme path.

More information

Distributed Accelerated Proximal Coordinate Gradient Methods

Distributed Accelerated Proximal Coordinate Gradient Methods Dstrbuted Accelerated Proxmal Coordate Gradet Methods Yog Re, Ju Zhu Ceter for Bo-Ispred Computg Research State Key Lab for Itell. Tech. & Systems Dept. of Comp. Sc. & Tech., TNLst Lab, Tsghua Uversty

More information

13. Parametric and Non-Parametric Uncertainties, Radial Basis Functions and Neural Network Approximations

13. Parametric and Non-Parametric Uncertainties, Radial Basis Functions and Neural Network Approximations Lecture 7 3. Parametrc ad No-Parametrc Ucertates, Radal Bass Fuctos ad Neural Network Approxmatos he parameter estmato algorthms descrbed prevous sectos were based o the assumpto that the system ucertates

More information

Randomized Dual Coordinate Ascent with Arbitrary Sampling

Randomized Dual Coordinate Ascent with Arbitrary Sampling Radomzed Dual Coordate Ascet wth Arbtrary Samplg Zheg Qu Peter Rchtárk Tog Zhag November 21, 2014 Abstract We study the problem of mmzg the average of a large umber of smooth covex fuctos pealzed wth a

More information

Maximum Likelihood Estimation

Maximum Likelihood Estimation Marquette Uverst Maxmum Lkelhood Estmato Dael B. Rowe, Ph.D. Professor Departmet of Mathematcs, Statstcs, ad Computer Scece Coprght 08 b Marquette Uverst Maxmum Lkelhood Estmato We have bee sag that ~

More information

Lecture 07: Poles and Zeros

Lecture 07: Poles and Zeros Lecture 07: Poles ad Zeros Defto of poles ad zeros The trasfer fucto provdes a bass for determg mportat system respose characterstcs wthout solvg the complete dfferetal equato. As defed, the trasfer fucto

More information

A nonsmooth Levenberg-Marquardt method for generalized complementarity problem

A nonsmooth Levenberg-Marquardt method for generalized complementarity problem ISSN 746-7659 Egla UK Joural of Iformato a Computg Scece Vol. 7 No. 4 0 pp. 67-7 A osmooth Leveberg-Marquart metho for geeralze complemetarty problem Shou-qag Du College of Mathematcs Qgao Uversty Qgao

More information

TESTS BASED ON MAXIMUM LIKELIHOOD

TESTS BASED ON MAXIMUM LIKELIHOOD ESE 5 Toy E. Smth. The Basc Example. TESTS BASED ON MAXIMUM LIKELIHOOD To llustrate the propertes of maxmum lkelhood estmates ad tests, we cosder the smplest possble case of estmatg the mea of the ormal

More information

Generalization of the Dissimilarity Measure of Fuzzy Sets

Generalization of the Dissimilarity Measure of Fuzzy Sets Iteratoal Mathematcal Forum 2 2007 o. 68 3395-3400 Geeralzato of the Dssmlarty Measure of Fuzzy Sets Faramarz Faghh Boformatcs Laboratory Naobotechology Research Ceter vesa Research Isttute CECR Tehra

More information

NP!= P. By Liu Ran. Table of Contents. The P vs. NP problem is a major unsolved problem in computer

NP!= P. By Liu Ran. Table of Contents. The P vs. NP problem is a major unsolved problem in computer NP!= P By Lu Ra Table of Cotets. Itroduce 2. Strategy 3. Prelmary theorem 4. Proof 5. Expla 6. Cocluso. Itroduce The P vs. NP problem s a major usolved problem computer scece. Iformally, t asks whether

More information

Transforms that are commonly used are separable

Transforms that are commonly used are separable Trasforms s Trasforms that are commoly used are separable Eamples: Two-dmesoal DFT DCT DST adamard We ca the use -D trasforms computg the D separable trasforms: Take -D trasform of the rows > rows ( )

More information

UNIT 2 SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS

UNIT 2 SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS Numercal Computg -I UNIT SOLUTION OF ALGEBRAIC AND TRANSCENDENTAL EQUATIONS Structure Page Nos..0 Itroducto 6. Objectves 7. Ital Approxmato to a Root 7. Bsecto Method 8.. Error Aalyss 9.4 Regula Fals Method

More information

Communication-Efficient Distributed Primal-Dual Algorithm for Saddle Point Problems

Communication-Efficient Distributed Primal-Dual Algorithm for Saddle Point Problems Commucato-Effcet Dstrbuted Prmal-Dual Algorthm for Saddle Pot Problems Yaodog Yu Nayag Techologcal Uversty ydyu@tu.edu.sg Sul Lu Nayag Techologcal Uversty lusl@tu.edu.sg So Jal Pa Nayag Techologcal Uversty

More information

Aitken delta-squared generalized Juncgk-type iterative procedure

Aitken delta-squared generalized Juncgk-type iterative procedure Atke delta-squared geeralzed Jucgk-type teratve procedure M. De la Se Isttute of Research ad Developmet of Processes. Uversty of Basque Coutry Campus of Leoa (Bzkaa) PO Box. 644- Blbao, 488- Blbao. SPAIN

More information

A Strong Convergence Theorem for a Proximal-Type. Algorithm in Re exive Banach Spaces

A Strong Convergence Theorem for a Proximal-Type. Algorithm in Re exive Banach Spaces A Strog Covergece Theorem for a Proxmal-Type Algorthm Re exve Baach Spaces Smeo Rech ad Shoham Sabach Abstract. We establsh a strog covergece theorem for a proxmal-type algorthm whch approxmates (commo)

More information

A Penalty Function Algorithm with Objective Parameters and Constraint Penalty Parameter for Multi-Objective Programming

A Penalty Function Algorithm with Objective Parameters and Constraint Penalty Parameter for Multi-Objective Programming Aerca Joural of Operatos Research, 4, 4, 33-339 Publshed Ole Noveber 4 ScRes http://wwwscrporg/oural/aor http://ddoorg/436/aor4463 A Pealty Fucto Algorth wth Obectve Paraeters ad Costrat Pealty Paraeter

More information

A tighter lower bound on the circuit size of the hardest Boolean functions

A tighter lower bound on the circuit size of the hardest Boolean functions Electroc Colloquum o Computatoal Complexty, Report No. 86 2011) A tghter lower boud o the crcut sze of the hardest Boolea fuctos Masak Yamamoto Abstract I [IPL2005], Fradse ad Mlterse mproved bouds o the

More information

4 Inner Product Spaces

4 Inner Product Spaces 11.MH1 LINEAR ALGEBRA Summary Notes 4 Ier Product Spaces Ier product s the abstracto to geeral vector spaces of the famlar dea of the scalar product of two vectors or 3. I what follows, keep these key

More information

Point Estimation: definition of estimators

Point Estimation: definition of estimators Pot Estmato: defto of estmators Pot estmator: ay fucto W (X,..., X ) of a data sample. The exercse of pot estmato s to use partcular fuctos of the data order to estmate certa ukow populato parameters.

More information

QR Factorization and Singular Value Decomposition COS 323

QR Factorization and Singular Value Decomposition COS 323 QR Factorzato ad Sgular Value Decomposto COS 33 Why Yet Aother Method? How do we solve least-squares wthout currg codto-squarg effect of ormal equatos (A T A A T b) whe A s sgular, fat, or otherwse poorly-specfed?

More information

C-1: Aerodynamics of Airfoils 1 C-2: Aerodynamics of Airfoils 2 C-3: Panel Methods C-4: Thin Airfoil Theory

C-1: Aerodynamics of Airfoils 1 C-2: Aerodynamics of Airfoils 2 C-3: Panel Methods C-4: Thin Airfoil Theory ROAD MAP... AE301 Aerodyamcs I UNIT C: 2-D Arfols C-1: Aerodyamcs of Arfols 1 C-2: Aerodyamcs of Arfols 2 C-3: Pael Methods C-4: Th Arfol Theory AE301 Aerodyamcs I Ut C-3: Lst of Subects Problem Solutos?

More information

The Lie Algebra of Smooth Sections of a T-bundle

The Lie Algebra of Smooth Sections of a T-bundle IST Iteratoal Joural of Egeerg Scece, Vol 7, No3-4, 6, Page 8-85 The Le Algera of Smooth Sectos of a T-udle Nadafhah ad H R Salm oghaddam Astract: I ths artcle, we geeralze the cocept of the Le algera

More information

Rademacher Complexity. Examples

Rademacher Complexity. Examples Algorthmc Foudatos of Learg Lecture 3 Rademacher Complexty. Examples Lecturer: Patrck Rebesch Verso: October 16th 018 3.1 Itroducto I the last lecture we troduced the oto of Rademacher complexty ad showed

More information

AN EULER-MC LAURIN FORMULA FOR INFINITE DIMENSIONAL SPACES

AN EULER-MC LAURIN FORMULA FOR INFINITE DIMENSIONAL SPACES AN EULER-MC LAURIN FORMULA FOR INFINITE DIMENSIONAL SPACES Jose Javer Garca Moreta Graduate Studet of Physcs ( Sold State ) at UPV/EHU Address: P.O 6 890 Portugalete, Vzcaya (Spa) Phoe: (00) 3 685 77 16

More information

Chapter 14 Logistic Regression Models

Chapter 14 Logistic Regression Models Chapter 4 Logstc Regresso Models I the lear regresso model X β + ε, there are two types of varables explaatory varables X, X,, X k ad study varable y These varables ca be measured o a cotuous scale as

More information

TRIANGULAR MEMBERSHIP FUNCTIONS FOR SOLVING SINGLE AND MULTIOBJECTIVE FUZZY LINEAR PROGRAMMING PROBLEM.

TRIANGULAR MEMBERSHIP FUNCTIONS FOR SOLVING SINGLE AND MULTIOBJECTIVE FUZZY LINEAR PROGRAMMING PROBLEM. Abbas Iraq Joural of SceceVol 53No 12012 Pp. 125-129 TRIANGULAR MEMBERSHIP FUNCTIONS FOR SOLVING SINGLE AND MULTIOBJECTIVE FUZZY LINEAR PROGRAMMING PROBLEM. Iraq Tarq Abbas Departemet of Mathematc College

More information

ρ < 1 be five real numbers. The

ρ < 1 be five real numbers. The Lecture o BST 63: Statstcal Theory I Ku Zhag, /0/006 Revew for the prevous lecture Deftos: covarace, correlato Examples: How to calculate covarace ad correlato Theorems: propertes of correlato ad covarace

More information

A Fixed Point Method for Convex Systems

A Fixed Point Method for Convex Systems Appled Mathematcs 3 37-333 http://ddoor/436/am3389 Publshed Ole October (http://wwwscrpor/oural/am) A Fed Pot Method for Cove Systems Morteza Kmae Farzad Rahpeyma Departmet of Mathematcs Raz Uversty Kermashah

More information

Evaluation of uncertainty in measurements

Evaluation of uncertainty in measurements Evaluato of ucertaty measuremets Laboratory of Physcs I Faculty of Physcs Warsaw Uversty of Techology Warszawa, 05 Itroducto The am of the measuremet s to determe the measured value. Thus, the measuremet

More information

Solution of General Dual Fuzzy Linear Systems. Using ABS Algorithm

Solution of General Dual Fuzzy Linear Systems. Using ABS Algorithm Appled Mathematcal Sceces, Vol 6, 0, o 4, 63-7 Soluto of Geeral Dual Fuzzy Lear Systems Usg ABS Algorthm M A Farborz Aragh * ad M M ossezadeh Departmet of Mathematcs, Islamc Azad Uversty Cetral ehra Brach,

More information

EVALUATION OF FUNCTIONAL INTEGRALS BY MEANS OF A SERIES AND THE METHOD OF BOREL TRANSFORM

EVALUATION OF FUNCTIONAL INTEGRALS BY MEANS OF A SERIES AND THE METHOD OF BOREL TRANSFORM EVALUATION OF FUNCTIONAL INTEGRALS BY MEANS OF A SERIES AND THE METHOD OF BOREL TRANSFORM Jose Javer Garca Moreta Ph. D research studet at the UPV/EHU (Uversty of Basque coutry) Departmet of Theoretcal

More information

Research and Simulation of FECG Signal Blind Separation Algorithm Based on Gradient Method

Research and Simulation of FECG Signal Blind Separation Algorithm Based on Gradient Method Research Joural of Appled Sceces, Egeerg ad Techology 4(6): 2707-27, 202 ISSN: 2040-7467 Maxwell Scetfc Orgazato, 202 Submtted: March 23, 202 Accepted: Aprl 20, 202 Publshed: August 5, 202 Research ad

More information

Solving Constrained Flow-Shop Scheduling. Problems with Three Machines

Solving Constrained Flow-Shop Scheduling. Problems with Three Machines It J Cotemp Math Sceces, Vol 5, 2010, o 19, 921-929 Solvg Costraed Flow-Shop Schedulg Problems wth Three Maches P Pada ad P Rajedra Departmet of Mathematcs, School of Advaced Sceces, VIT Uversty, Vellore-632

More information

The Necessarily Efficient Point Method for Interval Molp Problems

The Necessarily Efficient Point Method for Interval Molp Problems ISS 6-69 Eglad K Joural of Iformato ad omputg Scece Vol. o. 9 pp. - The ecessarly Effcet Pot Method for Iterval Molp Problems Hassa Mshmast eh ad Marzeh Alezhad + Mathematcs Departmet versty of Ssta ad

More information

Relations to Other Statistical Methods Statistical Data Analysis with Positive Definite Kernels

Relations to Other Statistical Methods Statistical Data Analysis with Positive Definite Kernels Relatos to Other Statstcal Methods Statstcal Data Aalyss wth Postve Defte Kerels Kej Fukuzu Isttute of Statstcal Matheatcs, ROIS Departet of Statstcal Scece, Graduate Uversty for Advaced Studes October

More information

Strong Convergence of Weighted Averaged Approximants of Asymptotically Nonexpansive Mappings in Banach Spaces without Uniform Convexity

Strong Convergence of Weighted Averaged Approximants of Asymptotically Nonexpansive Mappings in Banach Spaces without Uniform Convexity BULLETIN of the MALAYSIAN MATHEMATICAL SCIENCES SOCIETY Bull. Malays. Math. Sc. Soc. () 7 (004), 5 35 Strog Covergece of Weghted Averaged Appromats of Asymptotcally Noepasve Mappgs Baach Spaces wthout

More information

Minimization of Unconstrained Nonpolynomial Large-Scale Optimization Problems Using Conjugate Gradient Method Via Exact Line Search

Minimization of Unconstrained Nonpolynomial Large-Scale Optimization Problems Using Conjugate Gradient Method Via Exact Line Search Amerca Joural of Mechacal ad Materals Eeer 207; (): 0-4 http://wwwscecepublshroupcom/j/ajmme do: 0648/jajmme207003 Mmzato of Ucostraed Nopolyomal Lare-Scale Optmzato Problems Us Cojuate Gradet Method Va

More information

Gender Classification from ECG Signal Analysis using Least Square Support Vector Machine

Gender Classification from ECG Signal Analysis using Least Square Support Vector Machine Amerca Joural of Sgal Processg, (5): 45-49 DOI:.593/.asp.5.8 Geder Classfcato from ECG Sgal Aalyss usg Least Square Support Vector Mache Raesh Ku. rpathy,*, Ashutosh Acharya, Sumt Kumar Choudhary Departmet

More information

A Remark on the Uniform Convergence of Some Sequences of Functions

A Remark on the Uniform Convergence of Some Sequences of Functions Advaces Pure Mathematcs 05 5 57-533 Publshed Ole July 05 ScRes. http://www.scrp.org/joural/apm http://dx.do.org/0.436/apm.05.59048 A Remark o the Uform Covergece of Some Sequeces of Fuctos Guy Degla Isttut

More information

Difference of Convex Functions Optimization Methods: Algorithm of Minimum Maximal Network Flow Problem with Time-Windows

Difference of Convex Functions Optimization Methods: Algorithm of Minimum Maximal Network Flow Problem with Time-Windows Iteratoal Joural of Sceces: Basc ad Appled Research (IJSBAR) ISSN 307-453 (Prt & Ole) http://gssrr.org/dex.php?oural=jouralofbascadappled ---------------------------------------------------------------------------------------------------------------------------

More information

Taylor s Series and Interpolation. Interpolation & Curve-fitting. CIS Interpolation. Basic Scenario. Taylor Series interpolates at a specific

Taylor s Series and Interpolation. Interpolation & Curve-fitting. CIS Interpolation. Basic Scenario. Taylor Series interpolates at a specific CIS 54 - Iterpolato Roger Crawfs Basc Scearo We are able to prod some fucto, but do ot kow what t really s. Ths gves us a lst of data pots: [x,f ] f(x) f f + x x + August 2, 25 OSU/CIS 54 3 Taylor s Seres

More information

Lebesgue Measure of Generalized Cantor Set

Lebesgue Measure of Generalized Cantor Set Aals of Pure ad Appled Mathematcs Vol., No.,, -8 ISSN: -8X P), -888ole) Publshed o 8 May www.researchmathsc.org Aals of Lebesgue Measure of Geeralzed ator Set Md. Jahurul Islam ad Md. Shahdul Islam Departmet

More information

Unit 9. The Tangent Bundle

Unit 9. The Tangent Bundle Ut 9. The Taget Budle ========================================================================================== ---------- The taget sace of a submafold of R, detfcato of taget vectors wth dervatos at

More information

KLT Tracker. Alignment. 1. Detect Harris corners in the first frame. 2. For each Harris corner compute motion between consecutive frames

KLT Tracker. Alignment. 1. Detect Harris corners in the first frame. 2. For each Harris corner compute motion between consecutive frames KLT Tracker Tracker. Detect Harrs corers the frst frame 2. For each Harrs corer compute moto betwee cosecutve frames (Algmet). 3. Lk moto vectors successve frames to get a track 4. Itroduce ew Harrs pots

More information

Fitting: Deformable contours

Fitting: Deformable contours //0 Recap so far: Groupg ad Fttg Fttg: Deformable cotours Moday, Feb Prof. UT-Aust Goal: move from array of pxel values (or flter outputs to a collecto of regos, objects, ad shapes. Groupg: Pxels vs. regos

More information

Study on a Fire Detection System Based on Support Vector Machine

Study on a Fire Detection System Based on Support Vector Machine Sesors & Trasducers, Vol. 8, Issue, November 04, pp. 57-6 Sesors & Trasducers 04 by IFSA Publshg, S. L. http://www.sesorsportal.com Study o a Fre Detecto System Based o Support Vector Mache Ye Xaotg, Wu

More information

Linear Regression Linear Regression with Shrinkage. Some slides are due to Tommi Jaakkola, MIT AI Lab

Linear Regression Linear Regression with Shrinkage. Some slides are due to Tommi Jaakkola, MIT AI Lab Lear Regresso Lear Regresso th Shrkage Some sldes are due to Tomm Jaakkola, MIT AI Lab Itroducto The goal of regresso s to make quattatve real valued predctos o the bass of a vector of features or attrbutes.

More information

3D Geometry for Computer Graphics. Lesson 2: PCA & SVD

3D Geometry for Computer Graphics. Lesson 2: PCA & SVD 3D Geometry for Computer Graphcs Lesso 2: PCA & SVD Last week - egedecomposto We wat to lear how the matrx A works: A 2 Last week - egedecomposto If we look at arbtrary vectors, t does t tell us much.

More information

Estimation of Stress- Strength Reliability model using finite mixture of exponential distributions

Estimation of Stress- Strength Reliability model using finite mixture of exponential distributions Iteratoal Joural of Computatoal Egeerg Research Vol, 0 Issue, Estmato of Stress- Stregth Relablty model usg fte mxture of expoetal dstrbutos K.Sadhya, T.S.Umamaheswar Departmet of Mathematcs, Lal Bhadur

More information

DIFFERENTIAL GEOMETRIC APPROACH TO HAMILTONIAN MECHANICS

DIFFERENTIAL GEOMETRIC APPROACH TO HAMILTONIAN MECHANICS DIFFERENTIAL GEOMETRIC APPROACH TO HAMILTONIAN MECHANICS Course Project: Classcal Mechacs (PHY 40) Suja Dabholkar (Y430) Sul Yeshwath (Y444). Itroducto Hamltoa mechacs s geometry phase space. It deals

More information

Part 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971))

Part 4b Asymptotic Results for MRR2 using PRESS. Recall that the PRESS statistic is a special type of cross validation procedure (see Allen (1971)) art 4b Asymptotc Results for MRR usg RESS Recall that the RESS statstc s a specal type of cross valdato procedure (see Alle (97)) partcular to the regresso problem ad volves fdg Y $,, the estmate at the

More information

Complete Convergence and Some Maximal Inequalities for Weighted Sums of Random Variables

Complete Convergence and Some Maximal Inequalities for Weighted Sums of Random Variables Joural of Sceces, Islamc Republc of Ira 8(4): -6 (007) Uversty of Tehra, ISSN 06-04 http://sceces.ut.ac.r Complete Covergece ad Some Maxmal Iequaltes for Weghted Sums of Radom Varables M. Am,,* H.R. Nl

More information

Global solution of optimization problems with signomial parts

Global solution of optimization problems with signomial parts Dscrete Optmzato 5 (2008) 08 20 www.elsever.com/locate/dsopt Global soluto of optmzato problems wth sgomal parts ay Pör a, Kaj-Mkael Björk b,, Tapo Westerlud c a Sector for Techology ad Commucato at Swedsh

More information

X X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then

X X X E[ ] E X E X. is the ()m n where the ( i,)th. j element is the mean of the ( i,)th., then Secto 5 Vectors of Radom Varables Whe workg wth several radom varables,,..., to arrage them vector form x, t s ofte coveet We ca the make use of matrx algebra to help us orgaze ad mapulate large umbers

More information

-Pareto Optimality for Nondifferentiable Multiobjective Programming via Penalty Function

-Pareto Optimality for Nondifferentiable Multiobjective Programming via Penalty Function JOURNAL OF MATHEMATICAL ANALYSIS AND APPLICATIONS 198, 248261 1996 ARTICLE NO. 0080 -Pareto Otalty for Nodfferetable Multobectve Prograg va Pealty Fucto J. C. Lu Secto of Matheatcs, Natoal Uersty Prearatory

More information

CIS 800/002 The Algorithmic Foundations of Data Privacy October 13, Lecture 9. Database Update Algorithms: Multiplicative Weights

CIS 800/002 The Algorithmic Foundations of Data Privacy October 13, Lecture 9. Database Update Algorithms: Multiplicative Weights CIS 800/002 The Algorthmc Foudatos of Data Prvacy October 13, 2011 Lecturer: Aaro Roth Lecture 9 Scrbe: Aaro Roth Database Update Algorthms: Multplcatve Weghts We ll recall aga) some deftos from last tme:

More information

Integral Equation Methods. Jacob White. Thanks to Deepak Ramaswamy, Michal Rewienski, Xin Wang and Karen Veroy

Integral Equation Methods. Jacob White. Thanks to Deepak Ramaswamy, Michal Rewienski, Xin Wang and Karen Veroy Itroducto to Smulato - Lecture 22 Itegral Equato ethods Jacob Whte Thaks to Deepak Ramaswamy, chal Rewesk, X Wag ad Kare Veroy Outle Itegral Equato ethods Exteror versus teror problems Start wth usg pot

More information

15-381: Artificial Intelligence. Regression and neural networks (NN)

15-381: Artificial Intelligence. Regression and neural networks (NN) 5-38: Artfcal Itellece Reresso ad eural etorks NN) Mmck the bra I the earl das of AI there as a lot of terest develop models that ca mmc huma thk. Whle o oe ke eactl ho the bra orks ad, eve thouh there

More information

9 U-STATISTICS. Eh =(m!) 1 Eh(X (1),..., X (m ) ) i.i.d

9 U-STATISTICS. Eh =(m!) 1 Eh(X (1),..., X (m ) ) i.i.d 9 U-STATISTICS Suppose,,..., are P P..d. wth CDF F. Our goal s to estmate the expectato t (P)=Eh(,,..., m ). Note that ths expectato requres more tha oe cotrast to E, E, or Eh( ). Oe example s E or P((,

More information