Introduction to Optimization Techniques. How to Solve Equations
|
|
- Lawrence Merritt
- 6 years ago
- Views:
Transcription
1 Itroductio to Optimizatio Techiques How to Solve Equatios
2 Iterative Methods of Optimizatio Iterative methods of optimizatio Solutio of the oliear equatios resultig form a optimizatio problem is usually beyod aalytic tractability ca be treated by computer methods. Two basic approaches for resolvig complex optimizatio problems by umerical techiques:. Formulate the ecessary coditios describig the optimal solutio ad solve these equatios umerically. 2. Bypass the formulatio of the ecessary coditios ad implemet a direct iterative search for the optimum. 2
3 Successive Approximatio Successive Approximatio Equatio of the form x T( x) x : fixed poit of the trasformatio T ivariat uder T For successive vectors x T( x), uder appropriate coditios the sequece { } coverges to a solutio of the origial equatio. x Defiitio Let S be a subset of a ormed space X ad let T be a trasformatio mappig S ito S. The T is said to be a cotractio mappig if there is a, 0 such that T( x ) T( x ) x x for all x, x S
4 Cotractio Mappig Theorem Theorem (Cotractio Mappig Theorem) If T is a cotractio mappig o a closed subset S of a Baach space, there is a uique vector x0 S satisfyig x0 T( x0). Furthermore, x0 ca be obtaied by the method successive approximatio startig from a arbitrary iitial vector i S. Proof. (Existece ad Uiqueess) First we show existece of. Select a arbitrary elemet x S. Defie the sequece { x } by the formula x T( x). The x x T( x) T( x ) x x. Therefore x x x 2 x. It follows that x x x x x x x x p p p p p2 p2 p3 x2 x k x2 x x2 x k 0. x 0 4
5 Cotractio Mappig Theorem { x } is a Cauchy sequece. Sice S is a closed subset of a complete space, a elemet x S such that x x. 0 0 We ow show that x T( x ). We have 0 0 x T( x ) x x x T( x ) x x x T( x ) x x x x 0 0 ca be made arbitrary small by appropriate choice of. Uiqueess of x0 Assume that ad y are fixed poits. The x0 0 x0 y0 T( x0) T( y0) x0 y0 x0 y0 5
6 Successive Approximatio f ( x) x x 3 x 2 x Successive approximatio. 6
7 Iterative Methods of Optimizatio Newto s Method A iterative techique for solvig a equatio of the form Px ( ). Theorem Let X ad Y be Baach spaces ad let P be a mappig from X to Y. Assume further that. P is twice Frechet differetiable ad that P( x) K. 2. There is a poit x X such that p P( x ) has a bouded iverse with p, p [ P( x )]. 3. The costat h K satisfies h. 2 The the sequece x x p [ P( x)] exists for all ad coverges to a solutio of Px ( ). p 7
8 Iterative Methods of Optimizatio Descet Methods Direct approach for optimizatio problems: Iteratio i such a way as to decrease the cost fuctioal cotiuously from oe step to the ext. Geeral iteratio formula x x p p : scalar step size : directio vector is selected to miimize f ( x p ) is arraged so that f( x p ) f( x ) for small positive is take as the smallest positive root of the equatio d f ( x p) 0 d 8
9 Iterative Methods of Optimizatio f icreasig x 2 p x The descet process i X Newto s method for optimizatio problems [ f ( )] f ( ) x x x x 9
10 Iterative Methods of Optimizatio Steepest Descet Method of steepest descet: most widely used descet procedure for miimizig a fuctioal f p Directioal vector at a give poit : egative of the gradiet of f at Ex. Miimizatio of a quadratic fuctioal f ( x) [ x, Qx] 2[ x, b] x x where Q is self-adjoit positive-defiite f ( x ) Assume costats m [ xqx, ] if xθ [ xx, ] M [ xqx, ] sup xθ [ xx, ] 0 m, M 0
11 Iterative Methods of Optimizatio f x 0 is miimized by a uique vector : Qx 0 b x: approximate solutio r b Qx: residual of the approximatio f ( x) 2r Method of steepest descet x x r r b Qx is chose to miimize f ( x ) f ( x ) [ x r, Q( x r)] 2[ x r, b] 2 has a miimum at [ r, Qr ] 2 [ r, r ] [ x, Qx ] 2[ x, b] [, ] r r [ r, Qr ]
12 Iterative Methods of Optimizatio Steepest descet method [, ] () x r r x r [ r, Qr ] r bqx Theorem For ay x X the sequece { x } defied by () coverges (i orm) to the uique solutio x of Qx b. Furthermore, defiig F( x) [ xx, Q( xx )] the rate of covergece satisfies m [ y, y] F( x) F( x) m m M where y x0 x. 2
13 Cojugate Directio Method Cojugate Directio Method Problem of miimizig a quadratic fuctioal o a Hilbert space Miimum orm problem i a Hilbert space. Normal equatio ca be solved by machiery of orthogoalizatio, the Gram-Schmidt procedure, ad Fourier series philosophy of cojugate directio method Quadratic objective fuctioal f f ( x) [ x, Qx] 2[ x, b] Q : self-adjoit liear operator [ xqx, ] M [ xx, ] [ xqx, ] m[ xx, ] M, m 0, x H 3
14 Cojugate Directio Method x 0 Uique vector miimizig f = Uique solutio of the equatio Qx b. Formulatio of a miimum orm problem New ier product: equivalet to miimizig [ xy, ] [ xqx, ] mi f ( x) [ x, Qx] 2[ x, b] Q 2 xx [ xx, Q( xx )] 0 Q 0 0 Suppose we have ( or ca geerate) a sequece of vectors { p, p2, } orthogoal w.r.t. ier product [,] Q Q-orthogoal sequece, a sequece of cojugate directios 4
15 Cojugate Directio Method x 0 The vector ca be expaded i a Fourier series w.r.t. x 0 k p k k { p, p, } 2 the -th partial sum x p k k k x x 0 Q is miimized over the subspace S{ p, p,, p } 2 If { p } is complete, coverges to x. Fourier coefficiet k x 0 [ p, x ] [ p, Qx ] [ p, b] k i 0 Q i 0 i 5
16 Cojugate Directio Method Theorem (Method of Cojugate Directios) Let {p i } be a sequece i H such that [ pi, Qp j] 0, i j, ad such that the closed liear subspace geerated by the sequece is H. The for ay x H the sequece geerated by the recursio x x p [ p, r] [ p, Qp] r bqx satisfies [ r, pk] 0, k,2,, ad x x 0 which is the uique solutio of Qx b. 6
17 Proof Cojugate Directio Method Defie y x x, y θ. The, [ p, bqx Qy ] [ p, y y ] y y p y p 0 Q [ p, Qp] [ p, p] Q Sice y S{ p, p,, p } ad {p i } is Q-orthogoal, [ p, y ] 0. 2 [ p, y ] [ p, y ] y y p p 0 Q k 0 Q k [ p, p] Q k [ pk, pk] Q Q -th partial sum of a Fourier expasio of y 0 Covergece w.r.t Covergece w.r.t. Q y y x x 0 0 7
18 Cojugate Directio Method Orthogoality relatio [ r, pk] 0 follows from the fact that the error y y x x is Q-orthogoal to the subspace S{ p, p,, p }. 0 0 Ex Basic approximatio problem i a Hilbert space X. ˆ i i 2 Fid x a y i S{ y, y2,, y } which best approximates a give vector x. Normal equatio: Ga b G: Gram matrix of { y, y2,, y } b [ xy, ] i i Ga b is equivalet to the ucostraied miimizatio: T T mi aga2 ab, a a E 8
19 Cojugate Directio Method This problem ca be solved by the cojugate directio method. {p i } ca be costructed by G.S.P. k-th approximatio k a xˆ k k ai yi i best approximatio to x i the subspace S{ y, y,, y }. 2 k Cojugate directio method Solvig the origial approximatio problem by a Gram-Schmidt orthogoalizatio of the vectors { y, y2,, y}. 9
20 Cojugate Gradiet Method The Cojugate Gradiet Method Method of selectig directio vectors for miimizig the fuctioal f ( x) [ x, Qx] 2[ b, x]. Choose p = r = b - Qx p : directio of the egative gradiet of f at x. [, ] 2. Choose p r r Qp p, where r 2 = b - Qx [ p, Qp] p 2 is Q-orthogoal to p S{ p, p 2 } S{ r, r 2 } { r, r 2, } : sequece of egative gradiets { p, p, } : sequece of Q-orthogoalized versio of {r i } 2 20
21 3. recursive formula Cojugate Gradiet Method [ r, p ] x x p [ p, Qp] [ r, Qp ] p r p [ p, Qp] Theorem Let x H be give. Defie p = b - Qx ad (*) r b Qx x x p p r p [ r, p] [ r, Qp] where, [ p, Qp ] [ p, Qp ] The the sequece {x } coverges to x 0 = Q - b. 2
22 Proof Cojugate Gradiet Method Claim: this is a method of cojugate directios Assume the above claim is true for { pk} k, { xk} k. We shall show that it is true for oe more step. Sice α + is chose accordig to C.D.M., we must oly show that p + is Q-orthogoal to { }. [ p, Qp ] [ p, Qr ] [ p, Qp ] k k k p k k For k =, [ p, Qr ] [ p, Qp] 0 For k <, [ p, Qp ] 0 k [, ] p Qr [ p, Qp ] pk Qr Qpk r Qpk S p p2 pk [, ] [, ] {,,, } S{ p, p,, p } 2 22
23 Cojugate Gradiet Method For ay cojugate directio method [ r, ] 0,. pi i The method is a cojugate directio method. Next, we prove that the sequece {x } coverges to x 0. Defie x b Qx Q b Qx E( ) [, ( )] E( x ) E( x ) [ r, p ] (*) p r p [ r, p ] 0 [ r, p ] [ r, r ] [ r, r ] E( x ) E( x ) E( x ) [ r, Q r] 23
24 Cojugate Gradiet Method From (*) ad the Q-orthogoality of p ad p [ r, Qr] [ p, Qp] [ p, Qp ] By defiitio of m 3 [ r, Q r] [ p, Qp ] [ r, r ] m From, 2, 3, m E( x ) ( ) E( x) M E( x ) 0 r θ 24
25 Cojugate Gradiet Method Sigificat improvemet i the rate of covergece. For m[ xx, ] [ xqx, ] M[ xx, ], the covergece rate c 0 E( ) x x x m m c M For steepest descet, the best estimate 2 2 c x x0 E( x) m c The cojugate directio method coverges to the optimal solutio withi steps i a -dim quadratic problem. c 25
26 PARTAN Extesio of the cojugate directio method: applicable to the miimizatio of a o-quadratic fuctioal f. PARTAN : Method of PARallel TANget B A x 2 x p r x p Poit : lie( x, x 2) lie( x, B) A A r : egative gradiet of f at f( x ) : the miimum of f alog the x lie( x, B) 26
27 PARTAN PARTAN procedure for miimizig a arbitrary fuctioal f.. First miimize f alog the egative gradiet directio from. Fid the poit A. 2. Miimize f alog the lie determied by x ad A. Fid x Set x x. Go to Step. 2 For a quadratic fuctioals, PARTAN = Cojugate gradiet method - Rapid covergece i oquadratic fuctioals - No theoretical covergece aalysis is available. Methods for Solvig Costraied Problems Devisig computatioal procedure requires a lot of igeuity ad thorough familiarity with the basic priciples ad existig techiques of the area. x 27
28 Projectio Method Optimizatio problem with costraits. Use a descet method while remaiig withi the costrait regio. Cosider miimize Subject to f ( x) Ax b f : fuctioal o the Hilbert space X A : bouded liear operator : H Y b : fixed by A is assumed to have closed rage.. Start from a poit x { x Ax b} 2. Fid p (directio vector) from steepest descet or Newto s method 3. Project p oto the ullspace of A, N ( A). Fid the ew directio vector. g 28
29 4. Next poit x2 x g Sice g ( A), Projectio Method is chose from mi f ( x g) N 2 ( ) A x A x g Ax b x2 { x Ax b} : feasible solutio f( x ) * f x g R A ( ) ( ) g N ( A) f x g A A * ( ) R( ) N ( ) g f( x ) A * 29
30 g Projectio Method * is chose so that { ( ) } N ( A) g I A AA A f x * * [ ( ) ] ( ) A f x A At the solutio, f( x ) N ( A) x 0 f( x ) * f x0 R A ( ) ( ) N ( A) f( x ) A * 0 0 Necessary coditio i terms of the Lagrage multiplier 30
31 Primal-Dual Method Duality theorem : basis for optimizatio problem. Cosider a dual method for the covex problem: miimize f( x) subject to Gx ( ), x Assumig that the costrait is regular, this problem is equivalet to * max if { f( x) G( x), z } * z, x Defie the dual fuctioal * * ( z ) if{ f( x) G( x), z } x Equivalet dual problem * maximize ( z ) * subject to z 3
32 Primal-Dual Method Ex (Hildreth s Quadratic Programmig Procedure) Cosider T miimize xqx 2 subject to Ax c T bx x Q A b c : -vector : positive defiite matrix m : matrix : vector : m vector 32
33 Dual problem maximize subject to Primal-Dual Method 2 2 T T T P d b Q b T where P AQ A, d AQ bc Equivaletly miimize 2 subject to T T P Optimal solutio 0 x Q ( ba ) T 0 0 d 33
Introduction to Optimization Techniques
Itroductio to Optimizatio Techiques Basic Cocepts of Aalysis - Real Aalysis, Fuctioal Aalysis 1 Basic Cocepts of Aalysis Liear Vector Spaces Defiitio: A vector space X is a set of elemets called vectors
More informationDifferentiable Convex Functions
Differetiable Covex Fuctios The followig picture motivates Theorem 11. f ( x) f ( x) f '( x)( x x) ˆx x 1 Theorem 11 : Let f : R R be differetiable. The, f is covex o the covex set C R if, ad oly if for
More informationPAPER : IIT-JAM 2010
MATHEMATICS-MA (CODE A) Q.-Q.5: Oly oe optio is correct for each questio. Each questio carries (+6) marks for correct aswer ad ( ) marks for icorrect aswer.. Which of the followig coditios does NOT esure
More informationv = -!g(x 0 ) Ûg Ûx 1 Ûx 2 Ú If we work out the details in the partial derivatives, we get a pleasing result. n Ûx k, i x i - 2 b k
The Method of Steepest Descet This is the quadratic fuctio from to that is costructed to have a miimum at the x that solves the system A x = b: g(x) = - 2 I the method of steepest descet, we
More informationOptimization Methods MIT 2.098/6.255/ Final exam
Optimizatio Methods MIT 2.098/6.255/15.093 Fial exam Date Give: December 19th, 2006 P1. [30 pts] Classify the followig statemets as true or false. All aswers must be well-justified, either through a short
More information15.081J/6.251J Introduction to Mathematical Programming. Lecture 21: Primal Barrier Interior Point Algorithm
508J/65J Itroductio to Mathematical Programmig Lecture : Primal Barrier Iterior Poit Algorithm Outlie Barrier Methods Slide The Cetral Path 3 Approximatig the Cetral Path 4 The Primal Barrier Algorithm
More informationb i u x i U a i j u x i u x j
M ath 5 2 7 Fall 2 0 0 9 L ecture 1 9 N ov. 1 6, 2 0 0 9 ) S ecod- Order Elliptic Equatios: Weak S olutios 1. Defiitios. I this ad the followig two lectures we will study the boudary value problem Here
More informationReal Numbers R ) - LUB(B) may or may not belong to B. (Ex; B= { y: y = 1 x, - Note that A B LUB( A) LUB( B)
Real Numbers The least upper boud - Let B be ay subset of R B is bouded above if there is a k R such that x k for all x B - A real umber, k R is a uique least upper boud of B, ie k = LUB(B), if () k is
More informationMachine Learning for Data Science (CS 4786)
Machie Learig for Data Sciece CS 4786) Lecture & 3: Pricipal Compoet Aalysis The text i black outlies high level ideas. The text i blue provides simple mathematical details to derive or get to the algorithm
More informationCommon Coupled Fixed Point of Mappings Satisfying Rational Inequalities in Ordered Complex Valued Generalized Metric Spaces
IOSR Joural of Mathematics (IOSR-JM) e-issn: 78-578, p-issn:319-765x Volume 10, Issue 3 Ver II (May-Ju 014), PP 69-77 Commo Coupled Fixed Poit of Mappigs Satisfyig Ratioal Iequalities i Ordered Complex
More informationChapter 3 Inner Product Spaces. Hilbert Spaces
Chapter 3 Ier Product Spaces. Hilbert Spaces 3. Ier Product Spaces. Hilbert Spaces 3.- Defiitio. A ier product space is a vector space X with a ier product defied o X. A Hilbert space is a complete ier
More informationAbstract Vector Spaces. Abstract Vector Spaces
Astract Vector Spaces The process of astractio is critical i egieerig! Physical Device Data Storage Vector Space MRI machie Optical receiver 0 0 1 0 1 0 0 1 Icreasig astractio 6.1 Astract Vector Spaces
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More informationDefinitions and Theorems. where x are the decision variables. c, b, and a are constant coefficients.
Defiitios ad Theorems Remember the scalar form of the liear programmig problem, Miimize, Subject to, f(x) = c i x i a 1i x i = b 1 a mi x i = b m x i 0 i = 1,2,, where x are the decisio variables. c, b,
More informationMath Solutions to homework 6
Math 175 - Solutios to homework 6 Cédric De Groote November 16, 2017 Problem 1 (8.11 i the book): Let K be a compact Hermitia operator o a Hilbert space H ad let the kerel of K be {0}. Show that there
More informationSupport vector machine revisited
6.867 Machie learig, lecture 8 (Jaakkola) 1 Lecture topics: Support vector machie ad kerels Kerel optimizatio, selectio Support vector machie revisited Our task here is to first tur the support vector
More information10/2/ , 5.9, Jacob Hays Amit Pillay James DeFelice
0//008 Liear Discrimiat Fuctios Jacob Hays Amit Pillay James DeFelice 5.8, 5.9, 5. Miimum Squared Error Previous methods oly worked o liear separable cases, by lookig at misclassified samples to correct
More informationLinear Support Vector Machines
Liear Support Vector Machies David S. Roseberg The Support Vector Machie For a liear support vector machie (SVM), we use the hypothesis space of affie fuctios F = { f(x) = w T x + b w R d, b R } ad evaluate
More informationLinear Classifiers III
Uiversität Potsdam Istitut für Iformatik Lehrstuhl Maschielles Lere Liear Classifiers III Blaie Nelso, Tobias Scheffer Cotets Classificatio Problem Bayesia Classifier Decisio Liear Classifiers, MAP Models
More informationSolutions to home assignments (sketches)
Matematiska Istitutioe Peter Kumli 26th May 2004 TMA401 Fuctioal Aalysis MAN670 Applied Fuctioal Aalysis 4th quarter 2003/2004 All documet cocerig the course ca be foud o the course home page: http://www.math.chalmers.se/math/grudutb/cth/tma401/
More informationIntroduction to Machine Learning DIS10
CS 189 Fall 017 Itroductio to Machie Learig DIS10 1 Fu with Lagrage Multipliers (a) Miimize the fuctio such that f (x,y) = x + y x + y = 3. Solutio: The Lagragia is: L(x,y,λ) = x + y + λ(x + y 3) Takig
More informationLECTURE 8: ORTHOGONALITY (CHAPTER 5 IN THE BOOK)
LECTURE 8: ORTHOGONALITY (CHAPTER 5 IN THE BOOK) Everythig marked by is ot required by the course syllabus I this lecture, all vector spaces is over the real umber R. All vectors i R is viewed as a colum
More informationMachine Learning for Data Science (CS 4786)
Machie Learig for Data Sciece CS 4786) Lecture 9: Pricipal Compoet Aalysis The text i black outlies mai ideas to retai from the lecture. The text i blue give a deeper uderstadig of how we derive or get
More information5.1. The Rayleigh s quotient. Definition 49. Let A = A be a self-adjoint matrix. quotient is the function. R(x) = x,ax, for x = 0.
40 RODICA D. COSTIN 5. The Rayleigh s priciple ad the i priciple for the eigevalues of a self-adjoit matrix Eigevalues of self-adjoit matrices are easy to calculate. This sectio shows how this is doe usig
More informationPC5215 Numerical Recipes with Applications - Review Problems
PC55 Numerical Recipes with Applicatios - Review Problems Give the IEEE 754 sigle precisio bit patter (biary or he format) of the followig umbers: 0 0 05 00 0 00 Note that it has 8 bits for the epoet,
More informationLinear Elliptic PDE s Elliptic partial differential equations frequently arise out of conservation statements of the form
Liear Elliptic PDE s Elliptic partial differetial equatios frequetly arise out of coservatio statemets of the form B F d B Sdx B cotaied i bouded ope set U R. Here F, S deote respectively, the flux desity
More informationACO Comprehensive Exam 9 October 2007 Student code A. 1. Graph Theory
1. Graph Theory Prove that there exist o simple plaar triagulatio T ad two distict adjacet vertices x, y V (T ) such that x ad y are the oly vertices of T of odd degree. Do ot use the Four-Color Theorem.
More informationMarkov Decision Processes
Markov Decisio Processes Defiitios; Statioary policies; Value improvemet algorithm, Policy improvemet algorithm, ad liear programmig for discouted cost ad average cost criteria. Markov Decisio Processes
More informationON WELLPOSEDNESS QUADRATIC FUNCTION MINIMIZATION PROBLEM ON INTERSECTION OF TWO ELLIPSOIDS * M. JA]IMOVI], I. KRNI] 1.
Yugoslav Joural of Operatios Research 1 (00), Number 1, 49-60 ON WELLPOSEDNESS QUADRATIC FUNCTION MINIMIZATION PROBLEM ON INTERSECTION OF TWO ELLIPSOIDS M. JA]IMOVI], I. KRNI] Departmet of Mathematics
More information18.657: Mathematics of Machine Learning
8.657: Mathematics of Machie Learig Lecturer: Philippe Rigollet Lecture 0 Scribe: Ade Forrow Oct. 3, 05 Recall the followig defiitios from last time: Defiitio: A fuctio K : X X R is called a positive symmetric
More informationOptimally Sparse SVMs
A. Proof of Lemma 3. We here prove a lower boud o the umber of support vectors to achieve geeralizatio bouds of the form which we cosider. Importatly, this result holds ot oly for liear classifiers, but
More informationMath 210A Homework 1
Math 0A Homework Edward Burkard Exercise. a) State the defiitio of a aalytic fuctio. b) What are the relatioships betwee aalytic fuctios ad the Cauchy-Riema equatios? Solutio. a) A fuctio f : G C is called
More informationWe are mainly going to be concerned with power series in x, such as. (x)} converges - that is, lims N n
Review of Power Series, Power Series Solutios A power series i x - a is a ifiite series of the form c (x a) =c +c (x a)+(x a) +... We also call this a power series cetered at a. Ex. (x+) is cetered at
More informationSYMMETRIC POSITIVE SEMI-DEFINITE SOLUTIONS OF AX = B AND XC = D
Joural of Pure ad Alied Mathematics: Advaces ad Alicatios olume, Number, 009, Pages 99-07 SYMMERIC POSIIE SEMI-DEFINIE SOLUIONS OF AX B AND XC D School of Mathematics ad Physics Jiagsu Uiversity of Sciece
More informationBrief Review of Functions of Several Variables
Brief Review of Fuctios of Several Variables Differetiatio Differetiatio Recall, a fuctio f : R R is differetiable at x R if ( ) ( ) lim f x f x 0 exists df ( x) Whe this limit exists we call it or f(
More informationAssignment 1 : Real Numbers, Sequences. for n 1. Show that (x n ) converges. Further, by observing that x n+2 + x n+1
Assigmet : Real Numbers, Sequeces. Let A be a o-empty subset of R ad α R. Show that α = supa if ad oly if α is ot a upper boud of A but α + is a upper boud of A for every N. 2. Let y (, ) ad x (, ). Evaluate
More information5 Birkhoff s Ergodic Theorem
5 Birkhoff s Ergodic Theorem Amog the most useful of the various geeralizatios of KolmogorovâĂŹs strog law of large umbers are the ergodic theorems of Birkhoff ad Kigma, which exted the validity of the
More informationA General Iterative Scheme for Variational Inequality Problems and Fixed Point Problems
A Geeral Iterative Scheme for Variatioal Iequality Problems ad Fixed Poit Problems Wicha Khogtham Abstract We itroduce a geeral iterative scheme for fidig a commo of the set solutios of variatioal iequality
More informationAssignment 5: Solutions
McGill Uiversity Departmet of Mathematics ad Statistics MATH 54 Aalysis, Fall 05 Assigmet 5: Solutios. Let y be a ubouded sequece of positive umbers satisfyig y + > y for all N. Let x be aother sequece
More informationTHE SOLUTION OF NONLINEAR EQUATIONS f( x ) = 0.
THE SOLUTION OF NONLINEAR EQUATIONS f( ) = 0. Noliear Equatio Solvers Bracketig. Graphical. Aalytical Ope Methods Bisectio False Positio (Regula-Falsi) Fied poit iteratio Newto Raphso Secat The root of
More informationThe Method of Least Squares. To understand least squares fitting of data.
The Method of Least Squares KEY WORDS Curve fittig, least square GOAL To uderstad least squares fittig of data To uderstad the least squares solutio of icosistet systems of liear equatios 1 Motivatio Curve
More informationPRELIM PROBLEM SOLUTIONS
PRELIM PROBLEM SOLUTIONS THE GRAD STUDENTS + KEN Cotets. Complex Aalysis Practice Problems 2. 2. Real Aalysis Practice Problems 2. 4 3. Algebra Practice Problems 2. 8. Complex Aalysis Practice Problems
More informationPolynomials with Rational Roots that Differ by a Non-zero Constant. Generalities
Polyomials with Ratioal Roots that Differ by a No-zero Costat Philip Gibbs The problem of fidig two polyomials P(x) ad Q(x) of a give degree i a sigle variable x that have all ratioal roots ad differ by
More informationA Genetic Algorithm for Solving General System of Equations
A Geetic Algorithm for Solvig Geeral System of Equatios Győző Molárka, Edit Miletics Departmet of Mathematics, Szécheyi Istvá Uiversity, Győr, Hugary molarka@sze.hu, miletics@sze.hu Abstract: For solvig
More informationApply change-of-basis formula to rewrite x as a linear combination of eigenvectors v j.
Eigevalue-Eigevector Istructor: Nam Su Wag eigemcd Ay vector i real Euclidea space of dimesio ca be uiquely epressed as a liear combiatio of liearly idepedet vectors (ie, basis) g j, j,,, α g α g α g α
More informationSlide Set 13 Linear Model with Endogenous Regressors and the GMM estimator
Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday
More informationSection A assesses the Units Numerical Analysis 1 and 2 Section B assesses the Unit Mathematics for Applied Mathematics
X0/70 NATIONAL QUALIFICATIONS 005 MONDAY, MAY.00 PM 4.00 PM APPLIED MATHEMATICS ADVANCED HIGHER Numerical Aalysis Read carefully. Calculators may be used i this paper.. Cadidates should aswer all questios.
More informationNICK DUFRESNE. 1 1 p(x). To determine some formulas for the generating function of the Schröder numbers, r(x) = a(x) =
AN INTRODUCTION TO SCHRÖDER AND UNKNOWN NUMBERS NICK DUFRESNE Abstract. I this article we will itroduce two types of lattice paths, Schröder paths ad Ukow paths. We will examie differet properties of each,
More informationStatistical Inference Based on Extremum Estimators
T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0
More informationA Primal-Dual Interior-Point Filter Method for Nonlinear Semidefinite Programming
The 7th Iteratioal Symposium o Operatios Research ad Its Applicatios (ISORA 8) Lijiag, Chia, October 31 Novemver 3, 28 Copyright 28 ORSC & APORC, pp. 112 118 A Primal-Dual Iterior-Poit Filter Method for
More informationUnit 4: Polynomial and Rational Functions
48 Uit 4: Polyomial ad Ratioal Fuctios Polyomial Fuctios A polyomial fuctio y px ( ) is a fuctio of the form p( x) ax + a x + a x +... + ax + ax+ a 1 1 1 0 where a, a 1,..., a, a1, a0are real costats ad
More informationRiesz-Fischer Sequences and Lower Frame Bounds
Zeitschrift für Aalysis ud ihre Aweduge Joural for Aalysis ad its Applicatios Volume 1 (00), No., 305 314 Riesz-Fischer Sequeces ad Lower Frame Bouds P. Casazza, O. Christese, S. Li ad A. Lider Abstract.
More informationCS537. Numerical Analysis and Computing
CS57 Numerical Aalysis ad Computig Lecture Locatig Roots o Equatios Proessor Ju Zhag Departmet o Computer Sciece Uiversity o Ketucky Leigto KY 456-6 Jauary 9 9 What is the Root May physical system ca be
More information2 Banach spaces and Hilbert spaces
2 Baach spaces ad Hilbert spaces Tryig to do aalysis i the ratioal umbers is difficult for example cosider the set {x Q : x 2 2}. This set is o-empty ad bouded above but does ot have a least upper boud
More information1 Duality revisited. AM 221: Advanced Optimization Spring 2016
AM 22: Advaced Optimizatio Sprig 206 Prof. Yaro Siger Sectio 7 Wedesday, Mar. 9th Duality revisited I this sectio, we will give a slightly differet perspective o duality. optimizatio program: f(x) x R
More informationALGEBRAIC GEOMETRY COURSE NOTES, LECTURE 5: SINGULARITIES.
ALGEBRAIC GEOMETRY COURSE NOTES, LECTURE 5: SINGULARITIES. ANDREW SALCH 1. The Jacobia criterio for osigularity. You have probably oticed by ow that some poits o varieties are smooth i a sese somethig
More informationCHAPTER 1 SEQUENCES AND INFINITE SERIES
CHAPTER SEQUENCES AND INFINITE SERIES SEQUENCES AND INFINITE SERIES (0 meetigs) Sequeces ad limit of a sequece Mootoic ad bouded sequece Ifiite series of costat terms Ifiite series of positive terms Alteratig
More informationLyapunov Stability Analysis for Feedback Control Design
Copyright F.L. Lewis 008 All rights reserved Updated: uesday, November, 008 Lyapuov Stability Aalysis for Feedbac Cotrol Desig Lyapuov heorems Lyapuov Aalysis allows oe to aalyze the stability of cotiuous-time
More informationLecture Notes for Analysis Class
Lecture Notes for Aalysis Class Topological Spaces A topology for a set X is a collectio T of subsets of X such that: (a) X ad the empty set are i T (b) Uios of elemets of T are i T (c) Fiite itersectios
More informationw (1) ˆx w (1) x (1) /ρ and w (2) ˆx w (2) x (2) /ρ.
2 5. Weighted umber of late jobs 5.1. Release dates ad due dates: maximimizig the weight of o-time jobs Oce we add release dates, miimizig the umber of late jobs becomes a sigificatly harder problem. For
More informationNEW FAST CONVERGENT SEQUENCES OF EULER-MASCHERONI TYPE
UPB Sci Bull, Series A, Vol 79, Iss, 207 ISSN 22-7027 NEW FAST CONVERGENT SEQUENCES OF EULER-MASCHERONI TYPE Gabriel Bercu We itroduce two ew sequeces of Euler-Mascheroi type which have fast covergece
More informationTEACHER CERTIFICATION STUDY GUIDE
COMPETENCY 1. ALGEBRA SKILL 1.1 1.1a. ALGEBRAIC STRUCTURES Kow why the real ad complex umbers are each a field, ad that particular rigs are ot fields (e.g., itegers, polyomial rigs, matrix rigs) Algebra
More informationBeyond simple iteration of a single function, or even a finite sequence of functions, results
A Primer o the Elemetary Theory of Ifiite Compositios of Complex Fuctios Joh Gill Sprig 07 Abstract: Elemetary meas ot requirig the complex fuctios be holomorphic Theorem proofs are fairly simple ad are
More informationComplex Analysis Spring 2001 Homework I Solution
Complex Aalysis Sprig 2001 Homework I Solutio 1. Coway, Chapter 1, sectio 3, problem 3. Describe the set of poits satisfyig the equatio z a z + a = 2c, where c > 0 ad a R. To begi, we see from the triagle
More informationLecture 4 Conformal Mapping and Green s Theorem. 1. Let s try to solve the following problem by separation of variables
Lecture 4 Coformal Mappig ad Gree s Theorem Today s topics. Solvig electrostatic problems cotiued. Why separatio of variables does t always work 3. Coformal mappig 4. Gree s theorem The failure of separatio
More informationCS321. Numerical Analysis and Computing
CS Numerical Aalysis ad Computig Lecture Locatig Roots o Equatios Proessor Ju Zhag Departmet o Computer Sciece Uiversity o Ketucky Leigto KY 456-6 September 8 5 What is the Root May physical system ca
More informationNew Iterative Method for Variational Inclusion and Fixed Point Problems
Proceedigs of the World Cogress o Egieerig 04 Vol II, WCE 04, July - 4, 04, Lodo, U.K. Ne Iterative Method for Variatioal Iclusio ad Fixed Poit Problems Yaoaluck Khogtham Abstract We itroduce a iterative
More informationHomework 4. x n x X = f(x n x) +
Homework 4 1. Let X ad Y be ormed spaces, T B(X, Y ) ad {x } a sequece i X. If x x weakly, show that T x T x weakly. Solutio: We eed to show that g(t x) g(t x) g Y. It suffices to do this whe g Y = 1.
More informationSingular Continuous Measures by Michael Pejic 5/14/10
Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable
More informationHomework 2. Show that if h is a bounded sesquilinear form on the Hilbert spaces X and Y, then h has the representation
omework 2 1 Let X ad Y be ilbert spaces over C The a sesquiliear form h o X Y is a mappig h : X Y C such that for all x 1, x 2, x X, y 1, y 2, y Y ad all scalars α, β C we have (a) h(x 1 + x 2, y) h(x
More informationLecture 20. Brief Review of Gram-Schmidt and Gauss s Algorithm
8.409 A Algorithmist s Toolkit Nov. 9, 2009 Lecturer: Joatha Keler Lecture 20 Brief Review of Gram-Schmidt ad Gauss s Algorithm Our mai task of this lecture is to show a polyomial time algorithm which
More informationAre adaptive Mann iterations really adaptive?
MATHEMATICAL COMMUNICATIONS 399 Math. Commu., Vol. 4, No. 2, pp. 399-42 (2009) Are adaptive Ma iteratios really adaptive? Kamil S. Kazimierski, Departmet of Mathematics ad Computer Sciece, Uiversity of
More informationLinear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d
Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y
More informationA Proof of Birkhoff s Ergodic Theorem
A Proof of Birkhoff s Ergodic Theorem Joseph Hora September 2, 205 Itroductio I Fall 203, I was learig the basics of ergodic theory, ad I came across this theorem. Oe of my supervisors, Athoy Quas, showed
More informationState Space Representation
Optimal Cotrol, Guidace ad Estimatio Lecture 2 Overview of SS Approach ad Matrix heory Prof. Radhakat Padhi Dept. of Aerospace Egieerig Idia Istitute of Sciece - Bagalore State Space Represetatio Prof.
More information1 1 2 = show that: over variables x and y. [2 marks] Write down necessary conditions involving first and second-order partial derivatives for ( x0, y
Questio (a) A square matrix A= A is called positive defiite if the quadratic form waw > 0 for every o-zero vector w [Note: Here (.) deotes the traspose of a matrix or a vector]. Let 0 A = 0 = show that:
More informationA) is empty. B) is a finite set. C) can be a countably infinite set. D) can be an uncountable set.
M.A./M.Sc. (Mathematics) Etrace Examiatio 016-17 Max Time: hours Max Marks: 150 Istructios: There are 50 questios. Every questio has four choices of which exactly oe is correct. For correct aswer, 3 marks
More information6.003 Homework #3 Solutions
6.00 Homework # Solutios Problems. Complex umbers a. Evaluate the real ad imagiary parts of j j. π/ Real part = Imagiary part = 0 e Euler s formula says that j = e jπ/, so jπ/ j π/ j j = e = e. Thus the
More informationQuestions and answers, kernel part
Questios ad aswers, kerel part October 8, 205 Questios. Questio : properties of kerels, PCA, represeter theorem. [2 poits] Let F be a RK defied o some domai X, with feature map φ(x) x X ad reproducig kerel
More informationChapter 7 Isoperimetric problem
Chapter 7 Isoperimetric problem Recall that the isoperimetric problem (see the itroductio its coectio with ido s proble) is oe of the most classical problem of a shape optimizatio. It ca be formulated
More informationChapter 2 The Solution of Numerical Algebraic and Transcendental Equations
Chapter The Solutio of Numerical Algebraic ad Trascedetal Equatios Itroductio I this chapter we shall discuss some umerical methods for solvig algebraic ad trascedetal equatios. The equatio f( is said
More informationA new iterative algorithm for reconstructing a signal from its dyadic wavelet transform modulus maxima
ol 46 No 6 SCIENCE IN CHINA (Series F) December 3 A ew iterative algorithm for recostructig a sigal from its dyadic wavelet trasform modulus maxima ZHANG Zhuosheg ( u ), LIU Guizhog ( q) & LIU Feg ( )
More information5. Matrix exponentials and Von Neumann s theorem The matrix exponential. For an n n matrix X we define
5. Matrix expoetials ad Vo Neuma s theorem 5.1. The matrix expoetial. For a matrix X we defie e X = exp X = I + X + X2 2! +... = 0 X!. We assume that the etries are complex so that exp is well defied o
More informationOn the Variations of Some Well Known Fixed Point Theorem in Metric Spaces
Turkish Joural of Aalysis ad Number Theory, 205, Vol 3, No 2, 70-74 Available olie at http://pubssciepubcom/tjat/3/2/7 Sciece ad Educatio Publishig DOI:0269/tjat-3-2-7 O the Variatios of Some Well Kow
More informationReview Problems 1. ICME and MS&E Refresher Course September 19, 2011 B = C = AB = A = A 2 = A 3... C 2 = C 3 = =
Review Problems ICME ad MS&E Refresher Course September 9, 0 Warm-up problems. For the followig matrices A = 0 B = C = AB = 0 fid all powers A,A 3,(which is A times A),... ad B,B 3,... ad C,C 3,... Solutio:
More informationMathematics 3 Outcome 1. Vectors (9/10 pers) Lesson, Outline, Approach etc. This is page number 13. produced for TeeJay Publishers by Tom Strang
Vectors (9/0 pers) Mathematics 3 Outcome / Revise positio vector, PQ = q p, commuicative, associative, zero vector, multiplicatio by a scalar k, compoets, magitude, uit vector, (i, j, ad k) as well as
More informationProduct measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.
Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the
More informationEfficient GMM LECTURE 12 GMM II
DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet
More informationIterative Techniques for Solving Ax b -(3.8). Assume that the system has a unique solution. Let x be the solution. Then x A 1 b.
Iterative Techiques for Solvig Ax b -(8) Cosider solvig liear systems of them form: Ax b where A a ij, x x i, b b i Assume that the system has a uique solutio Let x be the solutio The x A b Jacobi ad Gauss-Seidel
More informationFind a formula for the exponential function whose graph is given , 1 2,16 1, 6
Math 4 Activity (Due by EOC Apr. ) Graph the followig epoetial fuctios by modifyig the graph of f. Fid the rage of each fuctio.. g. g. g 4. g. g 6. g Fid a formula for the epoetial fuctio whose graph is
More informationResearch Article Convergence Theorems for Finite Family of Multivalued Maps in Uniformly Convex Banach Spaces
Iteratioal Scholarly Research Network ISRN Mathematical Aalysis Volume 2011, Article ID 576108, 13 pages doi:10.5402/2011/576108 Research Article Covergece Theorems for Fiite Family of Multivalued Maps
More informationDiscretized Scheme Examining Convergence and Geometric Ratio Profiles for Control Problem Constrained by Evolution Equation
Joural of Mathematics ad Statistics 7 (: 6-, ISSN 549-644 Sciece Publicatios Discretized Scheme Examiig Covergece ad Geometric Ratio Profiles for Cotrol Problem Costraied by Evolutio Equatio O. Olotu Departmet
More informationFeedback in Iterative Algorithms
Feedback i Iterative Algorithms Charles Byre (Charles Byre@uml.edu), Departmet of Mathematical Scieces, Uiversity of Massachusetts Lowell, Lowell, MA 01854 October 17, 2005 Abstract Whe the oegative system
More information( ) (( ) ) ANSWERS TO EXERCISES IN APPENDIX B. Section B.1 VECTORS AND SETS. Exercise B.1-1: Convex sets. are convex, , hence. and. (a) Let.
Joh Riley 8 Jue 03 ANSWERS TO EXERCISES IN APPENDIX B Sectio B VECTORS AND SETS Exercise B-: Covex sets (a) Let 0 x, x X, X, hece 0 x, x X ad 0 x, x X Sice X ad X are covex, x X ad x X The x X X, which
More informationLeast-square inversion with inexact adjoints. Method of conjugate directions: A tutorial a
Least-square iversio with iexact adjoits. Method of cojugate directios: A tutorial a a Published i SEP Report, 92, 253-365 (1996) Sergey Fomel 1 ABSTRACT This tutorial describes the classic method of cojugate
More information6. Kalman filter implementation for linear algebraic equations. Karhunen-Loeve decomposition
6. Kalma filter implemetatio for liear algebraic equatios. Karhue-Loeve decompositio 6.1. Solvable liear algebraic systems. Probabilistic iterpretatio. Let A be a quadratic matrix (ot obligatory osigular.
More informationMATHEMATICAL SCIENCES PAPER-II
MATHEMATICAL SCIENCES PAPER-II. Let {x } ad {y } be two sequeces of real umbers. Prove or disprove each of the statemets :. If {x y } coverges, ad if {y } is coverget, the {x } is coverget.. {x + y } coverges
More informationMulti parameter proximal point algorithms
Multi parameter proximal poit algorithms Ogaeditse A. Boikayo a,b,, Gheorghe Moroşau a a Departmet of Mathematics ad its Applicatios Cetral Europea Uiversity Nador u. 9, H-1051 Budapest, Hugary b Departmet
More informationMachine Learning Brett Bernstein
Machie Learig Brett Berstei Week 2 Lecture: Cocept Check Exercises Starred problems are optioal. Excess Risk Decompositio 1. Let X = Y = {1, 2,..., 10}, A = {1,..., 10, 11} ad suppose the data distributio
More information1. By using truth tables prove that, for all statements P and Q, the statement
Author: Satiago Salazar Problems I: Mathematical Statemets ad Proofs. By usig truth tables prove that, for all statemets P ad Q, the statemet P Q ad its cotrapositive ot Q (ot P) are equivalet. I example.2.3
More information