Introduction to Optimization Techniques. How to Solve Equations

Size: px
Start display at page:

Download "Introduction to Optimization Techniques. How to Solve Equations"

Transcription

1 Itroductio to Optimizatio Techiques How to Solve Equatios

2 Iterative Methods of Optimizatio Iterative methods of optimizatio Solutio of the oliear equatios resultig form a optimizatio problem is usually beyod aalytic tractability ca be treated by computer methods. Two basic approaches for resolvig complex optimizatio problems by umerical techiques:. Formulate the ecessary coditios describig the optimal solutio ad solve these equatios umerically. 2. Bypass the formulatio of the ecessary coditios ad implemet a direct iterative search for the optimum. 2

3 Successive Approximatio Successive Approximatio Equatio of the form x T( x) x : fixed poit of the trasformatio T ivariat uder T For successive vectors x T( x), uder appropriate coditios the sequece { } coverges to a solutio of the origial equatio. x Defiitio Let S be a subset of a ormed space X ad let T be a trasformatio mappig S ito S. The T is said to be a cotractio mappig if there is a, 0 such that T( x ) T( x ) x x for all x, x S

4 Cotractio Mappig Theorem Theorem (Cotractio Mappig Theorem) If T is a cotractio mappig o a closed subset S of a Baach space, there is a uique vector x0 S satisfyig x0 T( x0). Furthermore, x0 ca be obtaied by the method successive approximatio startig from a arbitrary iitial vector i S. Proof. (Existece ad Uiqueess) First we show existece of. Select a arbitrary elemet x S. Defie the sequece { x } by the formula x T( x). The x x T( x) T( x ) x x. Therefore x x x 2 x. It follows that x x x x x x x x p p p p p2 p2 p3 x2 x k x2 x x2 x k 0. x 0 4

5 Cotractio Mappig Theorem { x } is a Cauchy sequece. Sice S is a closed subset of a complete space, a elemet x S such that x x. 0 0 We ow show that x T( x ). We have 0 0 x T( x ) x x x T( x ) x x x T( x ) x x x x 0 0 ca be made arbitrary small by appropriate choice of. Uiqueess of x0 Assume that ad y are fixed poits. The x0 0 x0 y0 T( x0) T( y0) x0 y0 x0 y0 5

6 Successive Approximatio f ( x) x x 3 x 2 x Successive approximatio. 6

7 Iterative Methods of Optimizatio Newto s Method A iterative techique for solvig a equatio of the form Px ( ). Theorem Let X ad Y be Baach spaces ad let P be a mappig from X to Y. Assume further that. P is twice Frechet differetiable ad that P( x) K. 2. There is a poit x X such that p P( x ) has a bouded iverse with p, p [ P( x )]. 3. The costat h K satisfies h. 2 The the sequece x x p [ P( x)] exists for all ad coverges to a solutio of Px ( ). p 7

8 Iterative Methods of Optimizatio Descet Methods Direct approach for optimizatio problems: Iteratio i such a way as to decrease the cost fuctioal cotiuously from oe step to the ext. Geeral iteratio formula x x p p : scalar step size : directio vector is selected to miimize f ( x p ) is arraged so that f( x p ) f( x ) for small positive is take as the smallest positive root of the equatio d f ( x p) 0 d 8

9 Iterative Methods of Optimizatio f icreasig x 2 p x The descet process i X Newto s method for optimizatio problems [ f ( )] f ( ) x x x x 9

10 Iterative Methods of Optimizatio Steepest Descet Method of steepest descet: most widely used descet procedure for miimizig a fuctioal f p Directioal vector at a give poit : egative of the gradiet of f at Ex. Miimizatio of a quadratic fuctioal f ( x) [ x, Qx] 2[ x, b] x x where Q is self-adjoit positive-defiite f ( x ) Assume costats m [ xqx, ] if xθ [ xx, ] M [ xqx, ] sup xθ [ xx, ] 0 m, M 0

11 Iterative Methods of Optimizatio f x 0 is miimized by a uique vector : Qx 0 b x: approximate solutio r b Qx: residual of the approximatio f ( x) 2r Method of steepest descet x x r r b Qx is chose to miimize f ( x ) f ( x ) [ x r, Q( x r)] 2[ x r, b] 2 has a miimum at [ r, Qr ] 2 [ r, r ] [ x, Qx ] 2[ x, b] [, ] r r [ r, Qr ]

12 Iterative Methods of Optimizatio Steepest descet method [, ] () x r r x r [ r, Qr ] r bqx Theorem For ay x X the sequece { x } defied by () coverges (i orm) to the uique solutio x of Qx b. Furthermore, defiig F( x) [ xx, Q( xx )] the rate of covergece satisfies m [ y, y] F( x) F( x) m m M where y x0 x. 2

13 Cojugate Directio Method Cojugate Directio Method Problem of miimizig a quadratic fuctioal o a Hilbert space Miimum orm problem i a Hilbert space. Normal equatio ca be solved by machiery of orthogoalizatio, the Gram-Schmidt procedure, ad Fourier series philosophy of cojugate directio method Quadratic objective fuctioal f f ( x) [ x, Qx] 2[ x, b] Q : self-adjoit liear operator [ xqx, ] M [ xx, ] [ xqx, ] m[ xx, ] M, m 0, x H 3

14 Cojugate Directio Method x 0 Uique vector miimizig f = Uique solutio of the equatio Qx b. Formulatio of a miimum orm problem New ier product: equivalet to miimizig [ xy, ] [ xqx, ] mi f ( x) [ x, Qx] 2[ x, b] Q 2 xx [ xx, Q( xx )] 0 Q 0 0 Suppose we have ( or ca geerate) a sequece of vectors { p, p2, } orthogoal w.r.t. ier product [,] Q Q-orthogoal sequece, a sequece of cojugate directios 4

15 Cojugate Directio Method x 0 The vector ca be expaded i a Fourier series w.r.t. x 0 k p k k { p, p, } 2 the -th partial sum x p k k k x x 0 Q is miimized over the subspace S{ p, p,, p } 2 If { p } is complete, coverges to x. Fourier coefficiet k x 0 [ p, x ] [ p, Qx ] [ p, b] k i 0 Q i 0 i 5

16 Cojugate Directio Method Theorem (Method of Cojugate Directios) Let {p i } be a sequece i H such that [ pi, Qp j] 0, i j, ad such that the closed liear subspace geerated by the sequece is H. The for ay x H the sequece geerated by the recursio x x p [ p, r] [ p, Qp] r bqx satisfies [ r, pk] 0, k,2,, ad x x 0 which is the uique solutio of Qx b. 6

17 Proof Cojugate Directio Method Defie y x x, y θ. The, [ p, bqx Qy ] [ p, y y ] y y p y p 0 Q [ p, Qp] [ p, p] Q Sice y S{ p, p,, p } ad {p i } is Q-orthogoal, [ p, y ] 0. 2 [ p, y ] [ p, y ] y y p p 0 Q k 0 Q k [ p, p] Q k [ pk, pk] Q Q -th partial sum of a Fourier expasio of y 0 Covergece w.r.t Covergece w.r.t. Q y y x x 0 0 7

18 Cojugate Directio Method Orthogoality relatio [ r, pk] 0 follows from the fact that the error y y x x is Q-orthogoal to the subspace S{ p, p,, p }. 0 0 Ex Basic approximatio problem i a Hilbert space X. ˆ i i 2 Fid x a y i S{ y, y2,, y } which best approximates a give vector x. Normal equatio: Ga b G: Gram matrix of { y, y2,, y } b [ xy, ] i i Ga b is equivalet to the ucostraied miimizatio: T T mi aga2 ab, a a E 8

19 Cojugate Directio Method This problem ca be solved by the cojugate directio method. {p i } ca be costructed by G.S.P. k-th approximatio k a xˆ k k ai yi i best approximatio to x i the subspace S{ y, y,, y }. 2 k Cojugate directio method Solvig the origial approximatio problem by a Gram-Schmidt orthogoalizatio of the vectors { y, y2,, y}. 9

20 Cojugate Gradiet Method The Cojugate Gradiet Method Method of selectig directio vectors for miimizig the fuctioal f ( x) [ x, Qx] 2[ b, x]. Choose p = r = b - Qx p : directio of the egative gradiet of f at x. [, ] 2. Choose p r r Qp p, where r 2 = b - Qx [ p, Qp] p 2 is Q-orthogoal to p S{ p, p 2 } S{ r, r 2 } { r, r 2, } : sequece of egative gradiets { p, p, } : sequece of Q-orthogoalized versio of {r i } 2 20

21 3. recursive formula Cojugate Gradiet Method [ r, p ] x x p [ p, Qp] [ r, Qp ] p r p [ p, Qp] Theorem Let x H be give. Defie p = b - Qx ad (*) r b Qx x x p p r p [ r, p] [ r, Qp] where, [ p, Qp ] [ p, Qp ] The the sequece {x } coverges to x 0 = Q - b. 2

22 Proof Cojugate Gradiet Method Claim: this is a method of cojugate directios Assume the above claim is true for { pk} k, { xk} k. We shall show that it is true for oe more step. Sice α + is chose accordig to C.D.M., we must oly show that p + is Q-orthogoal to { }. [ p, Qp ] [ p, Qr ] [ p, Qp ] k k k p k k For k =, [ p, Qr ] [ p, Qp] 0 For k <, [ p, Qp ] 0 k [, ] p Qr [ p, Qp ] pk Qr Qpk r Qpk S p p2 pk [, ] [, ] {,,, } S{ p, p,, p } 2 22

23 Cojugate Gradiet Method For ay cojugate directio method [ r, ] 0,. pi i The method is a cojugate directio method. Next, we prove that the sequece {x } coverges to x 0. Defie x b Qx Q b Qx E( ) [, ( )] E( x ) E( x ) [ r, p ] (*) p r p [ r, p ] 0 [ r, p ] [ r, r ] [ r, r ] E( x ) E( x ) E( x ) [ r, Q r] 23

24 Cojugate Gradiet Method From (*) ad the Q-orthogoality of p ad p [ r, Qr] [ p, Qp] [ p, Qp ] By defiitio of m 3 [ r, Q r] [ p, Qp ] [ r, r ] m From, 2, 3, m E( x ) ( ) E( x) M E( x ) 0 r θ 24

25 Cojugate Gradiet Method Sigificat improvemet i the rate of covergece. For m[ xx, ] [ xqx, ] M[ xx, ], the covergece rate c 0 E( ) x x x m m c M For steepest descet, the best estimate 2 2 c x x0 E( x) m c The cojugate directio method coverges to the optimal solutio withi steps i a -dim quadratic problem. c 25

26 PARTAN Extesio of the cojugate directio method: applicable to the miimizatio of a o-quadratic fuctioal f. PARTAN : Method of PARallel TANget B A x 2 x p r x p Poit : lie( x, x 2) lie( x, B) A A r : egative gradiet of f at f( x ) : the miimum of f alog the x lie( x, B) 26

27 PARTAN PARTAN procedure for miimizig a arbitrary fuctioal f.. First miimize f alog the egative gradiet directio from. Fid the poit A. 2. Miimize f alog the lie determied by x ad A. Fid x Set x x. Go to Step. 2 For a quadratic fuctioals, PARTAN = Cojugate gradiet method - Rapid covergece i oquadratic fuctioals - No theoretical covergece aalysis is available. Methods for Solvig Costraied Problems Devisig computatioal procedure requires a lot of igeuity ad thorough familiarity with the basic priciples ad existig techiques of the area. x 27

28 Projectio Method Optimizatio problem with costraits. Use a descet method while remaiig withi the costrait regio. Cosider miimize Subject to f ( x) Ax b f : fuctioal o the Hilbert space X A : bouded liear operator : H Y b : fixed by A is assumed to have closed rage.. Start from a poit x { x Ax b} 2. Fid p (directio vector) from steepest descet or Newto s method 3. Project p oto the ullspace of A, N ( A). Fid the ew directio vector. g 28

29 4. Next poit x2 x g Sice g ( A), Projectio Method is chose from mi f ( x g) N 2 ( ) A x A x g Ax b x2 { x Ax b} : feasible solutio f( x ) * f x g R A ( ) ( ) g N ( A) f x g A A * ( ) R( ) N ( ) g f( x ) A * 29

30 g Projectio Method * is chose so that { ( ) } N ( A) g I A AA A f x * * [ ( ) ] ( ) A f x A At the solutio, f( x ) N ( A) x 0 f( x ) * f x0 R A ( ) ( ) N ( A) f( x ) A * 0 0 Necessary coditio i terms of the Lagrage multiplier 30

31 Primal-Dual Method Duality theorem : basis for optimizatio problem. Cosider a dual method for the covex problem: miimize f( x) subject to Gx ( ), x Assumig that the costrait is regular, this problem is equivalet to * max if { f( x) G( x), z } * z, x Defie the dual fuctioal * * ( z ) if{ f( x) G( x), z } x Equivalet dual problem * maximize ( z ) * subject to z 3

32 Primal-Dual Method Ex (Hildreth s Quadratic Programmig Procedure) Cosider T miimize xqx 2 subject to Ax c T bx x Q A b c : -vector : positive defiite matrix m : matrix : vector : m vector 32

33 Dual problem maximize subject to Primal-Dual Method 2 2 T T T P d b Q b T where P AQ A, d AQ bc Equivaletly miimize 2 subject to T T P Optimal solutio 0 x Q ( ba ) T 0 0 d 33

Introduction to Optimization Techniques

Introduction to Optimization Techniques Itroductio to Optimizatio Techiques Basic Cocepts of Aalysis - Real Aalysis, Fuctioal Aalysis 1 Basic Cocepts of Aalysis Liear Vector Spaces Defiitio: A vector space X is a set of elemets called vectors

More information

Differentiable Convex Functions

Differentiable Convex Functions Differetiable Covex Fuctios The followig picture motivates Theorem 11. f ( x) f ( x) f '( x)( x x) ˆx x 1 Theorem 11 : Let f : R R be differetiable. The, f is covex o the covex set C R if, ad oly if for

More information

PAPER : IIT-JAM 2010

PAPER : IIT-JAM 2010 MATHEMATICS-MA (CODE A) Q.-Q.5: Oly oe optio is correct for each questio. Each questio carries (+6) marks for correct aswer ad ( ) marks for icorrect aswer.. Which of the followig coditios does NOT esure

More information

v = -!g(x 0 ) Ûg Ûx 1 Ûx 2 Ú If we work out the details in the partial derivatives, we get a pleasing result. n Ûx k, i x i - 2 b k

v = -!g(x 0 ) Ûg Ûx 1 Ûx 2 Ú If we work out the details in the partial derivatives, we get a pleasing result. n Ûx k, i x i - 2 b k The Method of Steepest Descet This is the quadratic fuctio from to that is costructed to have a miimum at the x that solves the system A x = b: g(x) = - 2 I the method of steepest descet, we

More information

Optimization Methods MIT 2.098/6.255/ Final exam

Optimization Methods MIT 2.098/6.255/ Final exam Optimizatio Methods MIT 2.098/6.255/15.093 Fial exam Date Give: December 19th, 2006 P1. [30 pts] Classify the followig statemets as true or false. All aswers must be well-justified, either through a short

More information

15.081J/6.251J Introduction to Mathematical Programming. Lecture 21: Primal Barrier Interior Point Algorithm

15.081J/6.251J Introduction to Mathematical Programming. Lecture 21: Primal Barrier Interior Point Algorithm 508J/65J Itroductio to Mathematical Programmig Lecture : Primal Barrier Iterior Poit Algorithm Outlie Barrier Methods Slide The Cetral Path 3 Approximatig the Cetral Path 4 The Primal Barrier Algorithm

More information

b i u x i U a i j u x i u x j

b i u x i U a i j u x i u x j M ath 5 2 7 Fall 2 0 0 9 L ecture 1 9 N ov. 1 6, 2 0 0 9 ) S ecod- Order Elliptic Equatios: Weak S olutios 1. Defiitios. I this ad the followig two lectures we will study the boudary value problem Here

More information

Real Numbers R ) - LUB(B) may or may not belong to B. (Ex; B= { y: y = 1 x, - Note that A B LUB( A) LUB( B)

Real Numbers R ) - LUB(B) may or may not belong to B. (Ex; B= { y: y = 1 x, - Note that A B LUB( A) LUB( B) Real Numbers The least upper boud - Let B be ay subset of R B is bouded above if there is a k R such that x k for all x B - A real umber, k R is a uique least upper boud of B, ie k = LUB(B), if () k is

More information

Machine Learning for Data Science (CS 4786)

Machine Learning for Data Science (CS 4786) Machie Learig for Data Sciece CS 4786) Lecture & 3: Pricipal Compoet Aalysis The text i black outlies high level ideas. The text i blue provides simple mathematical details to derive or get to the algorithm

More information

Common Coupled Fixed Point of Mappings Satisfying Rational Inequalities in Ordered Complex Valued Generalized Metric Spaces

Common Coupled Fixed Point of Mappings Satisfying Rational Inequalities in Ordered Complex Valued Generalized Metric Spaces IOSR Joural of Mathematics (IOSR-JM) e-issn: 78-578, p-issn:319-765x Volume 10, Issue 3 Ver II (May-Ju 014), PP 69-77 Commo Coupled Fixed Poit of Mappigs Satisfyig Ratioal Iequalities i Ordered Complex

More information

Chapter 3 Inner Product Spaces. Hilbert Spaces

Chapter 3 Inner Product Spaces. Hilbert Spaces Chapter 3 Ier Product Spaces. Hilbert Spaces 3. Ier Product Spaces. Hilbert Spaces 3.- Defiitio. A ier product space is a vector space X with a ier product defied o X. A Hilbert space is a complete ier

More information

Abstract Vector Spaces. Abstract Vector Spaces

Abstract Vector Spaces. Abstract Vector Spaces Astract Vector Spaces The process of astractio is critical i egieerig! Physical Device Data Storage Vector Space MRI machie Optical receiver 0 0 1 0 1 0 0 1 Icreasig astractio 6.1 Astract Vector Spaces

More information

Definition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.

Definition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4. 4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad

More information

Definitions and Theorems. where x are the decision variables. c, b, and a are constant coefficients.

Definitions and Theorems. where x are the decision variables. c, b, and a are constant coefficients. Defiitios ad Theorems Remember the scalar form of the liear programmig problem, Miimize, Subject to, f(x) = c i x i a 1i x i = b 1 a mi x i = b m x i 0 i = 1,2,, where x are the decisio variables. c, b,

More information

Math Solutions to homework 6

Math Solutions to homework 6 Math 175 - Solutios to homework 6 Cédric De Groote November 16, 2017 Problem 1 (8.11 i the book): Let K be a compact Hermitia operator o a Hilbert space H ad let the kerel of K be {0}. Show that there

More information

Support vector machine revisited

Support vector machine revisited 6.867 Machie learig, lecture 8 (Jaakkola) 1 Lecture topics: Support vector machie ad kerels Kerel optimizatio, selectio Support vector machie revisited Our task here is to first tur the support vector

More information

10/2/ , 5.9, Jacob Hays Amit Pillay James DeFelice

10/2/ , 5.9, Jacob Hays Amit Pillay James DeFelice 0//008 Liear Discrimiat Fuctios Jacob Hays Amit Pillay James DeFelice 5.8, 5.9, 5. Miimum Squared Error Previous methods oly worked o liear separable cases, by lookig at misclassified samples to correct

More information

Linear Support Vector Machines

Linear Support Vector Machines Liear Support Vector Machies David S. Roseberg The Support Vector Machie For a liear support vector machie (SVM), we use the hypothesis space of affie fuctios F = { f(x) = w T x + b w R d, b R } ad evaluate

More information

Linear Classifiers III

Linear Classifiers III Uiversität Potsdam Istitut für Iformatik Lehrstuhl Maschielles Lere Liear Classifiers III Blaie Nelso, Tobias Scheffer Cotets Classificatio Problem Bayesia Classifier Decisio Liear Classifiers, MAP Models

More information

Solutions to home assignments (sketches)

Solutions to home assignments (sketches) Matematiska Istitutioe Peter Kumli 26th May 2004 TMA401 Fuctioal Aalysis MAN670 Applied Fuctioal Aalysis 4th quarter 2003/2004 All documet cocerig the course ca be foud o the course home page: http://www.math.chalmers.se/math/grudutb/cth/tma401/

More information

Introduction to Machine Learning DIS10

Introduction to Machine Learning DIS10 CS 189 Fall 017 Itroductio to Machie Learig DIS10 1 Fu with Lagrage Multipliers (a) Miimize the fuctio such that f (x,y) = x + y x + y = 3. Solutio: The Lagragia is: L(x,y,λ) = x + y + λ(x + y 3) Takig

More information

LECTURE 8: ORTHOGONALITY (CHAPTER 5 IN THE BOOK)

LECTURE 8: ORTHOGONALITY (CHAPTER 5 IN THE BOOK) LECTURE 8: ORTHOGONALITY (CHAPTER 5 IN THE BOOK) Everythig marked by is ot required by the course syllabus I this lecture, all vector spaces is over the real umber R. All vectors i R is viewed as a colum

More information

Machine Learning for Data Science (CS 4786)

Machine Learning for Data Science (CS 4786) Machie Learig for Data Sciece CS 4786) Lecture 9: Pricipal Compoet Aalysis The text i black outlies mai ideas to retai from the lecture. The text i blue give a deeper uderstadig of how we derive or get

More information

5.1. The Rayleigh s quotient. Definition 49. Let A = A be a self-adjoint matrix. quotient is the function. R(x) = x,ax, for x = 0.

5.1. The Rayleigh s quotient. Definition 49. Let A = A be a self-adjoint matrix. quotient is the function. R(x) = x,ax, for x = 0. 40 RODICA D. COSTIN 5. The Rayleigh s priciple ad the i priciple for the eigevalues of a self-adjoit matrix Eigevalues of self-adjoit matrices are easy to calculate. This sectio shows how this is doe usig

More information

PC5215 Numerical Recipes with Applications - Review Problems

PC5215 Numerical Recipes with Applications - Review Problems PC55 Numerical Recipes with Applicatios - Review Problems Give the IEEE 754 sigle precisio bit patter (biary or he format) of the followig umbers: 0 0 05 00 0 00 Note that it has 8 bits for the epoet,

More information

Linear Elliptic PDE s Elliptic partial differential equations frequently arise out of conservation statements of the form

Linear Elliptic PDE s Elliptic partial differential equations frequently arise out of conservation statements of the form Liear Elliptic PDE s Elliptic partial differetial equatios frequetly arise out of coservatio statemets of the form B F d B Sdx B cotaied i bouded ope set U R. Here F, S deote respectively, the flux desity

More information

ACO Comprehensive Exam 9 October 2007 Student code A. 1. Graph Theory

ACO Comprehensive Exam 9 October 2007 Student code A. 1. Graph Theory 1. Graph Theory Prove that there exist o simple plaar triagulatio T ad two distict adjacet vertices x, y V (T ) such that x ad y are the oly vertices of T of odd degree. Do ot use the Four-Color Theorem.

More information

Markov Decision Processes

Markov Decision Processes Markov Decisio Processes Defiitios; Statioary policies; Value improvemet algorithm, Policy improvemet algorithm, ad liear programmig for discouted cost ad average cost criteria. Markov Decisio Processes

More information

ON WELLPOSEDNESS QUADRATIC FUNCTION MINIMIZATION PROBLEM ON INTERSECTION OF TWO ELLIPSOIDS * M. JA]IMOVI], I. KRNI] 1.

ON WELLPOSEDNESS QUADRATIC FUNCTION MINIMIZATION PROBLEM ON INTERSECTION OF TWO ELLIPSOIDS * M. JA]IMOVI], I. KRNI] 1. Yugoslav Joural of Operatios Research 1 (00), Number 1, 49-60 ON WELLPOSEDNESS QUADRATIC FUNCTION MINIMIZATION PROBLEM ON INTERSECTION OF TWO ELLIPSOIDS M. JA]IMOVI], I. KRNI] Departmet of Mathematics

More information

18.657: Mathematics of Machine Learning

18.657: Mathematics of Machine Learning 8.657: Mathematics of Machie Learig Lecturer: Philippe Rigollet Lecture 0 Scribe: Ade Forrow Oct. 3, 05 Recall the followig defiitios from last time: Defiitio: A fuctio K : X X R is called a positive symmetric

More information

Optimally Sparse SVMs

Optimally Sparse SVMs A. Proof of Lemma 3. We here prove a lower boud o the umber of support vectors to achieve geeralizatio bouds of the form which we cosider. Importatly, this result holds ot oly for liear classifiers, but

More information

Math 210A Homework 1

Math 210A Homework 1 Math 0A Homework Edward Burkard Exercise. a) State the defiitio of a aalytic fuctio. b) What are the relatioships betwee aalytic fuctios ad the Cauchy-Riema equatios? Solutio. a) A fuctio f : G C is called

More information

We are mainly going to be concerned with power series in x, such as. (x)} converges - that is, lims N n

We are mainly going to be concerned with power series in x, such as. (x)} converges - that is, lims N n Review of Power Series, Power Series Solutios A power series i x - a is a ifiite series of the form c (x a) =c +c (x a)+(x a) +... We also call this a power series cetered at a. Ex. (x+) is cetered at

More information

SYMMETRIC POSITIVE SEMI-DEFINITE SOLUTIONS OF AX = B AND XC = D

SYMMETRIC POSITIVE SEMI-DEFINITE SOLUTIONS OF AX = B AND XC = D Joural of Pure ad Alied Mathematics: Advaces ad Alicatios olume, Number, 009, Pages 99-07 SYMMERIC POSIIE SEMI-DEFINIE SOLUIONS OF AX B AND XC D School of Mathematics ad Physics Jiagsu Uiversity of Sciece

More information

Brief Review of Functions of Several Variables

Brief Review of Functions of Several Variables Brief Review of Fuctios of Several Variables Differetiatio Differetiatio Recall, a fuctio f : R R is differetiable at x R if ( ) ( ) lim f x f x 0 exists df ( x) Whe this limit exists we call it or f(

More information

Assignment 1 : Real Numbers, Sequences. for n 1. Show that (x n ) converges. Further, by observing that x n+2 + x n+1

Assignment 1 : Real Numbers, Sequences. for n 1. Show that (x n ) converges. Further, by observing that x n+2 + x n+1 Assigmet : Real Numbers, Sequeces. Let A be a o-empty subset of R ad α R. Show that α = supa if ad oly if α is ot a upper boud of A but α + is a upper boud of A for every N. 2. Let y (, ) ad x (, ). Evaluate

More information

5 Birkhoff s Ergodic Theorem

5 Birkhoff s Ergodic Theorem 5 Birkhoff s Ergodic Theorem Amog the most useful of the various geeralizatios of KolmogorovâĂŹs strog law of large umbers are the ergodic theorems of Birkhoff ad Kigma, which exted the validity of the

More information

A General Iterative Scheme for Variational Inequality Problems and Fixed Point Problems

A General Iterative Scheme for Variational Inequality Problems and Fixed Point Problems A Geeral Iterative Scheme for Variatioal Iequality Problems ad Fixed Poit Problems Wicha Khogtham Abstract We itroduce a geeral iterative scheme for fidig a commo of the set solutios of variatioal iequality

More information

Assignment 5: Solutions

Assignment 5: Solutions McGill Uiversity Departmet of Mathematics ad Statistics MATH 54 Aalysis, Fall 05 Assigmet 5: Solutios. Let y be a ubouded sequece of positive umbers satisfyig y + > y for all N. Let x be aother sequece

More information

THE SOLUTION OF NONLINEAR EQUATIONS f( x ) = 0.

THE SOLUTION OF NONLINEAR EQUATIONS f( x ) = 0. THE SOLUTION OF NONLINEAR EQUATIONS f( ) = 0. Noliear Equatio Solvers Bracketig. Graphical. Aalytical Ope Methods Bisectio False Positio (Regula-Falsi) Fied poit iteratio Newto Raphso Secat The root of

More information

The Method of Least Squares. To understand least squares fitting of data.

The Method of Least Squares. To understand least squares fitting of data. The Method of Least Squares KEY WORDS Curve fittig, least square GOAL To uderstad least squares fittig of data To uderstad the least squares solutio of icosistet systems of liear equatios 1 Motivatio Curve

More information

PRELIM PROBLEM SOLUTIONS

PRELIM PROBLEM SOLUTIONS PRELIM PROBLEM SOLUTIONS THE GRAD STUDENTS + KEN Cotets. Complex Aalysis Practice Problems 2. 2. Real Aalysis Practice Problems 2. 4 3. Algebra Practice Problems 2. 8. Complex Aalysis Practice Problems

More information

Polynomials with Rational Roots that Differ by a Non-zero Constant. Generalities

Polynomials with Rational Roots that Differ by a Non-zero Constant. Generalities Polyomials with Ratioal Roots that Differ by a No-zero Costat Philip Gibbs The problem of fidig two polyomials P(x) ad Q(x) of a give degree i a sigle variable x that have all ratioal roots ad differ by

More information

A Genetic Algorithm for Solving General System of Equations

A Genetic Algorithm for Solving General System of Equations A Geetic Algorithm for Solvig Geeral System of Equatios Győző Molárka, Edit Miletics Departmet of Mathematics, Szécheyi Istvá Uiversity, Győr, Hugary molarka@sze.hu, miletics@sze.hu Abstract: For solvig

More information

Apply change-of-basis formula to rewrite x as a linear combination of eigenvectors v j.

Apply change-of-basis formula to rewrite x as a linear combination of eigenvectors v j. Eigevalue-Eigevector Istructor: Nam Su Wag eigemcd Ay vector i real Euclidea space of dimesio ca be uiquely epressed as a liear combiatio of liearly idepedet vectors (ie, basis) g j, j,,, α g α g α g α

More information

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator

Slide Set 13 Linear Model with Endogenous Regressors and the GMM estimator Slide Set 13 Liear Model with Edogeous Regressors ad the GMM estimator Pietro Coretto pcoretto@uisa.it Ecoometrics Master i Ecoomics ad Fiace (MEF) Uiversità degli Studi di Napoli Federico II Versio: Friday

More information

Section A assesses the Units Numerical Analysis 1 and 2 Section B assesses the Unit Mathematics for Applied Mathematics

Section A assesses the Units Numerical Analysis 1 and 2 Section B assesses the Unit Mathematics for Applied Mathematics X0/70 NATIONAL QUALIFICATIONS 005 MONDAY, MAY.00 PM 4.00 PM APPLIED MATHEMATICS ADVANCED HIGHER Numerical Aalysis Read carefully. Calculators may be used i this paper.. Cadidates should aswer all questios.

More information

NICK DUFRESNE. 1 1 p(x). To determine some formulas for the generating function of the Schröder numbers, r(x) = a(x) =

NICK DUFRESNE. 1 1 p(x). To determine some formulas for the generating function of the Schröder numbers, r(x) = a(x) = AN INTRODUCTION TO SCHRÖDER AND UNKNOWN NUMBERS NICK DUFRESNE Abstract. I this article we will itroduce two types of lattice paths, Schröder paths ad Ukow paths. We will examie differet properties of each,

More information

Statistical Inference Based on Extremum Estimators

Statistical Inference Based on Extremum Estimators T. Rotheberg Fall, 2007 Statistical Iferece Based o Extremum Estimators Itroductio Suppose 0, the true value of a p-dimesioal parameter, is kow to lie i some subset S R p : Ofte we choose to estimate 0

More information

A Primal-Dual Interior-Point Filter Method for Nonlinear Semidefinite Programming

A Primal-Dual Interior-Point Filter Method for Nonlinear Semidefinite Programming The 7th Iteratioal Symposium o Operatios Research ad Its Applicatios (ISORA 8) Lijiag, Chia, October 31 Novemver 3, 28 Copyright 28 ORSC & APORC, pp. 112 118 A Primal-Dual Iterior-Poit Filter Method for

More information

Unit 4: Polynomial and Rational Functions

Unit 4: Polynomial and Rational Functions 48 Uit 4: Polyomial ad Ratioal Fuctios Polyomial Fuctios A polyomial fuctio y px ( ) is a fuctio of the form p( x) ax + a x + a x +... + ax + ax+ a 1 1 1 0 where a, a 1,..., a, a1, a0are real costats ad

More information

Riesz-Fischer Sequences and Lower Frame Bounds

Riesz-Fischer Sequences and Lower Frame Bounds Zeitschrift für Aalysis ud ihre Aweduge Joural for Aalysis ad its Applicatios Volume 1 (00), No., 305 314 Riesz-Fischer Sequeces ad Lower Frame Bouds P. Casazza, O. Christese, S. Li ad A. Lider Abstract.

More information

CS537. Numerical Analysis and Computing

CS537. Numerical Analysis and Computing CS57 Numerical Aalysis ad Computig Lecture Locatig Roots o Equatios Proessor Ju Zhag Departmet o Computer Sciece Uiversity o Ketucky Leigto KY 456-6 Jauary 9 9 What is the Root May physical system ca be

More information

2 Banach spaces and Hilbert spaces

2 Banach spaces and Hilbert spaces 2 Baach spaces ad Hilbert spaces Tryig to do aalysis i the ratioal umbers is difficult for example cosider the set {x Q : x 2 2}. This set is o-empty ad bouded above but does ot have a least upper boud

More information

1 Duality revisited. AM 221: Advanced Optimization Spring 2016

1 Duality revisited. AM 221: Advanced Optimization Spring 2016 AM 22: Advaced Optimizatio Sprig 206 Prof. Yaro Siger Sectio 7 Wedesday, Mar. 9th Duality revisited I this sectio, we will give a slightly differet perspective o duality. optimizatio program: f(x) x R

More information

ALGEBRAIC GEOMETRY COURSE NOTES, LECTURE 5: SINGULARITIES.

ALGEBRAIC GEOMETRY COURSE NOTES, LECTURE 5: SINGULARITIES. ALGEBRAIC GEOMETRY COURSE NOTES, LECTURE 5: SINGULARITIES. ANDREW SALCH 1. The Jacobia criterio for osigularity. You have probably oticed by ow that some poits o varieties are smooth i a sese somethig

More information

CHAPTER 1 SEQUENCES AND INFINITE SERIES

CHAPTER 1 SEQUENCES AND INFINITE SERIES CHAPTER SEQUENCES AND INFINITE SERIES SEQUENCES AND INFINITE SERIES (0 meetigs) Sequeces ad limit of a sequece Mootoic ad bouded sequece Ifiite series of costat terms Ifiite series of positive terms Alteratig

More information

Lyapunov Stability Analysis for Feedback Control Design

Lyapunov Stability Analysis for Feedback Control Design Copyright F.L. Lewis 008 All rights reserved Updated: uesday, November, 008 Lyapuov Stability Aalysis for Feedbac Cotrol Desig Lyapuov heorems Lyapuov Aalysis allows oe to aalyze the stability of cotiuous-time

More information

Lecture Notes for Analysis Class

Lecture Notes for Analysis Class Lecture Notes for Aalysis Class Topological Spaces A topology for a set X is a collectio T of subsets of X such that: (a) X ad the empty set are i T (b) Uios of elemets of T are i T (c) Fiite itersectios

More information

w (1) ˆx w (1) x (1) /ρ and w (2) ˆx w (2) x (2) /ρ.

w (1) ˆx w (1) x (1) /ρ and w (2) ˆx w (2) x (2) /ρ. 2 5. Weighted umber of late jobs 5.1. Release dates ad due dates: maximimizig the weight of o-time jobs Oce we add release dates, miimizig the umber of late jobs becomes a sigificatly harder problem. For

More information

NEW FAST CONVERGENT SEQUENCES OF EULER-MASCHERONI TYPE

NEW FAST CONVERGENT SEQUENCES OF EULER-MASCHERONI TYPE UPB Sci Bull, Series A, Vol 79, Iss, 207 ISSN 22-7027 NEW FAST CONVERGENT SEQUENCES OF EULER-MASCHERONI TYPE Gabriel Bercu We itroduce two ew sequeces of Euler-Mascheroi type which have fast covergece

More information

TEACHER CERTIFICATION STUDY GUIDE

TEACHER CERTIFICATION STUDY GUIDE COMPETENCY 1. ALGEBRA SKILL 1.1 1.1a. ALGEBRAIC STRUCTURES Kow why the real ad complex umbers are each a field, ad that particular rigs are ot fields (e.g., itegers, polyomial rigs, matrix rigs) Algebra

More information

Beyond simple iteration of a single function, or even a finite sequence of functions, results

Beyond simple iteration of a single function, or even a finite sequence of functions, results A Primer o the Elemetary Theory of Ifiite Compositios of Complex Fuctios Joh Gill Sprig 07 Abstract: Elemetary meas ot requirig the complex fuctios be holomorphic Theorem proofs are fairly simple ad are

More information

Complex Analysis Spring 2001 Homework I Solution

Complex Analysis Spring 2001 Homework I Solution Complex Aalysis Sprig 2001 Homework I Solutio 1. Coway, Chapter 1, sectio 3, problem 3. Describe the set of poits satisfyig the equatio z a z + a = 2c, where c > 0 ad a R. To begi, we see from the triagle

More information

Lecture 4 Conformal Mapping and Green s Theorem. 1. Let s try to solve the following problem by separation of variables

Lecture 4 Conformal Mapping and Green s Theorem. 1. Let s try to solve the following problem by separation of variables Lecture 4 Coformal Mappig ad Gree s Theorem Today s topics. Solvig electrostatic problems cotiued. Why separatio of variables does t always work 3. Coformal mappig 4. Gree s theorem The failure of separatio

More information

CS321. Numerical Analysis and Computing

CS321. Numerical Analysis and Computing CS Numerical Aalysis ad Computig Lecture Locatig Roots o Equatios Proessor Ju Zhag Departmet o Computer Sciece Uiversity o Ketucky Leigto KY 456-6 September 8 5 What is the Root May physical system ca

More information

New Iterative Method for Variational Inclusion and Fixed Point Problems

New Iterative Method for Variational Inclusion and Fixed Point Problems Proceedigs of the World Cogress o Egieerig 04 Vol II, WCE 04, July - 4, 04, Lodo, U.K. Ne Iterative Method for Variatioal Iclusio ad Fixed Poit Problems Yaoaluck Khogtham Abstract We itroduce a iterative

More information

Homework 4. x n x X = f(x n x) +

Homework 4. x n x X = f(x n x) + Homework 4 1. Let X ad Y be ormed spaces, T B(X, Y ) ad {x } a sequece i X. If x x weakly, show that T x T x weakly. Solutio: We eed to show that g(t x) g(t x) g Y. It suffices to do this whe g Y = 1.

More information

Singular Continuous Measures by Michael Pejic 5/14/10

Singular Continuous Measures by Michael Pejic 5/14/10 Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable

More information

Homework 2. Show that if h is a bounded sesquilinear form on the Hilbert spaces X and Y, then h has the representation

Homework 2. Show that if h is a bounded sesquilinear form on the Hilbert spaces X and Y, then h has the representation omework 2 1 Let X ad Y be ilbert spaces over C The a sesquiliear form h o X Y is a mappig h : X Y C such that for all x 1, x 2, x X, y 1, y 2, y Y ad all scalars α, β C we have (a) h(x 1 + x 2, y) h(x

More information

Lecture 20. Brief Review of Gram-Schmidt and Gauss s Algorithm

Lecture 20. Brief Review of Gram-Schmidt and Gauss s Algorithm 8.409 A Algorithmist s Toolkit Nov. 9, 2009 Lecturer: Joatha Keler Lecture 20 Brief Review of Gram-Schmidt ad Gauss s Algorithm Our mai task of this lecture is to show a polyomial time algorithm which

More information

Are adaptive Mann iterations really adaptive?

Are adaptive Mann iterations really adaptive? MATHEMATICAL COMMUNICATIONS 399 Math. Commu., Vol. 4, No. 2, pp. 399-42 (2009) Are adaptive Ma iteratios really adaptive? Kamil S. Kazimierski, Departmet of Mathematics ad Computer Sciece, Uiversity of

More information

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d

Linear regression. Daniel Hsu (COMS 4771) (y i x T i β)2 2πσ. 2 2σ 2. 1 n. (x T i β y i ) 2. 1 ˆβ arg min. β R n d Liear regressio Daiel Hsu (COMS 477) Maximum likelihood estimatio Oe of the simplest liear regressio models is the followig: (X, Y ),..., (X, Y ), (X, Y ) are iid radom pairs takig values i R d R, ad Y

More information

A Proof of Birkhoff s Ergodic Theorem

A Proof of Birkhoff s Ergodic Theorem A Proof of Birkhoff s Ergodic Theorem Joseph Hora September 2, 205 Itroductio I Fall 203, I was learig the basics of ergodic theory, ad I came across this theorem. Oe of my supervisors, Athoy Quas, showed

More information

State Space Representation

State Space Representation Optimal Cotrol, Guidace ad Estimatio Lecture 2 Overview of SS Approach ad Matrix heory Prof. Radhakat Padhi Dept. of Aerospace Egieerig Idia Istitute of Sciece - Bagalore State Space Represetatio Prof.

More information

1 1 2 = show that: over variables x and y. [2 marks] Write down necessary conditions involving first and second-order partial derivatives for ( x0, y

1 1 2 = show that: over variables x and y. [2 marks] Write down necessary conditions involving first and second-order partial derivatives for ( x0, y Questio (a) A square matrix A= A is called positive defiite if the quadratic form waw > 0 for every o-zero vector w [Note: Here (.) deotes the traspose of a matrix or a vector]. Let 0 A = 0 = show that:

More information

A) is empty. B) is a finite set. C) can be a countably infinite set. D) can be an uncountable set.

A) is empty. B) is a finite set. C) can be a countably infinite set. D) can be an uncountable set. M.A./M.Sc. (Mathematics) Etrace Examiatio 016-17 Max Time: hours Max Marks: 150 Istructios: There are 50 questios. Every questio has four choices of which exactly oe is correct. For correct aswer, 3 marks

More information

6.003 Homework #3 Solutions

6.003 Homework #3 Solutions 6.00 Homework # Solutios Problems. Complex umbers a. Evaluate the real ad imagiary parts of j j. π/ Real part = Imagiary part = 0 e Euler s formula says that j = e jπ/, so jπ/ j π/ j j = e = e. Thus the

More information

Questions and answers, kernel part

Questions and answers, kernel part Questios ad aswers, kerel part October 8, 205 Questios. Questio : properties of kerels, PCA, represeter theorem. [2 poits] Let F be a RK defied o some domai X, with feature map φ(x) x X ad reproducig kerel

More information

Chapter 7 Isoperimetric problem

Chapter 7 Isoperimetric problem Chapter 7 Isoperimetric problem Recall that the isoperimetric problem (see the itroductio its coectio with ido s proble) is oe of the most classical problem of a shape optimizatio. It ca be formulated

More information

Chapter 2 The Solution of Numerical Algebraic and Transcendental Equations

Chapter 2 The Solution of Numerical Algebraic and Transcendental Equations Chapter The Solutio of Numerical Algebraic ad Trascedetal Equatios Itroductio I this chapter we shall discuss some umerical methods for solvig algebraic ad trascedetal equatios. The equatio f( is said

More information

A new iterative algorithm for reconstructing a signal from its dyadic wavelet transform modulus maxima

A new iterative algorithm for reconstructing a signal from its dyadic wavelet transform modulus maxima ol 46 No 6 SCIENCE IN CHINA (Series F) December 3 A ew iterative algorithm for recostructig a sigal from its dyadic wavelet trasform modulus maxima ZHANG Zhuosheg ( u ), LIU Guizhog ( q) & LIU Feg ( )

More information

5. Matrix exponentials and Von Neumann s theorem The matrix exponential. For an n n matrix X we define

5. Matrix exponentials and Von Neumann s theorem The matrix exponential. For an n n matrix X we define 5. Matrix expoetials ad Vo Neuma s theorem 5.1. The matrix expoetial. For a matrix X we defie e X = exp X = I + X + X2 2! +... = 0 X!. We assume that the etries are complex so that exp is well defied o

More information

On the Variations of Some Well Known Fixed Point Theorem in Metric Spaces

On the Variations of Some Well Known Fixed Point Theorem in Metric Spaces Turkish Joural of Aalysis ad Number Theory, 205, Vol 3, No 2, 70-74 Available olie at http://pubssciepubcom/tjat/3/2/7 Sciece ad Educatio Publishig DOI:0269/tjat-3-2-7 O the Variatios of Some Well Kow

More information

Review Problems 1. ICME and MS&E Refresher Course September 19, 2011 B = C = AB = A = A 2 = A 3... C 2 = C 3 = =

Review Problems 1. ICME and MS&E Refresher Course September 19, 2011 B = C = AB = A = A 2 = A 3... C 2 = C 3 = = Review Problems ICME ad MS&E Refresher Course September 9, 0 Warm-up problems. For the followig matrices A = 0 B = C = AB = 0 fid all powers A,A 3,(which is A times A),... ad B,B 3,... ad C,C 3,... Solutio:

More information

Mathematics 3 Outcome 1. Vectors (9/10 pers) Lesson, Outline, Approach etc. This is page number 13. produced for TeeJay Publishers by Tom Strang

Mathematics 3 Outcome 1. Vectors (9/10 pers) Lesson, Outline, Approach etc. This is page number 13. produced for TeeJay Publishers by Tom Strang Vectors (9/0 pers) Mathematics 3 Outcome / Revise positio vector, PQ = q p, commuicative, associative, zero vector, multiplicatio by a scalar k, compoets, magitude, uit vector, (i, j, ad k) as well as

More information

Product measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.

Product measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014. Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the

More information

Efficient GMM LECTURE 12 GMM II

Efficient GMM LECTURE 12 GMM II DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet

More information

Iterative Techniques for Solving Ax b -(3.8). Assume that the system has a unique solution. Let x be the solution. Then x A 1 b.

Iterative Techniques for Solving Ax b -(3.8). Assume that the system has a unique solution. Let x be the solution. Then x A 1 b. Iterative Techiques for Solvig Ax b -(8) Cosider solvig liear systems of them form: Ax b where A a ij, x x i, b b i Assume that the system has a uique solutio Let x be the solutio The x A b Jacobi ad Gauss-Seidel

More information

Find a formula for the exponential function whose graph is given , 1 2,16 1, 6

Find a formula for the exponential function whose graph is given , 1 2,16 1, 6 Math 4 Activity (Due by EOC Apr. ) Graph the followig epoetial fuctios by modifyig the graph of f. Fid the rage of each fuctio.. g. g. g 4. g. g 6. g Fid a formula for the epoetial fuctio whose graph is

More information

Research Article Convergence Theorems for Finite Family of Multivalued Maps in Uniformly Convex Banach Spaces

Research Article Convergence Theorems for Finite Family of Multivalued Maps in Uniformly Convex Banach Spaces Iteratioal Scholarly Research Network ISRN Mathematical Aalysis Volume 2011, Article ID 576108, 13 pages doi:10.5402/2011/576108 Research Article Covergece Theorems for Fiite Family of Multivalued Maps

More information

Discretized Scheme Examining Convergence and Geometric Ratio Profiles for Control Problem Constrained by Evolution Equation

Discretized Scheme Examining Convergence and Geometric Ratio Profiles for Control Problem Constrained by Evolution Equation Joural of Mathematics ad Statistics 7 (: 6-, ISSN 549-644 Sciece Publicatios Discretized Scheme Examiig Covergece ad Geometric Ratio Profiles for Cotrol Problem Costraied by Evolutio Equatio O. Olotu Departmet

More information

Feedback in Iterative Algorithms

Feedback in Iterative Algorithms Feedback i Iterative Algorithms Charles Byre (Charles Byre@uml.edu), Departmet of Mathematical Scieces, Uiversity of Massachusetts Lowell, Lowell, MA 01854 October 17, 2005 Abstract Whe the oegative system

More information

( ) (( ) ) ANSWERS TO EXERCISES IN APPENDIX B. Section B.1 VECTORS AND SETS. Exercise B.1-1: Convex sets. are convex, , hence. and. (a) Let.

( ) (( ) ) ANSWERS TO EXERCISES IN APPENDIX B. Section B.1 VECTORS AND SETS. Exercise B.1-1: Convex sets. are convex, , hence. and. (a) Let. Joh Riley 8 Jue 03 ANSWERS TO EXERCISES IN APPENDIX B Sectio B VECTORS AND SETS Exercise B-: Covex sets (a) Let 0 x, x X, X, hece 0 x, x X ad 0 x, x X Sice X ad X are covex, x X ad x X The x X X, which

More information

Least-square inversion with inexact adjoints. Method of conjugate directions: A tutorial a

Least-square inversion with inexact adjoints. Method of conjugate directions: A tutorial a Least-square iversio with iexact adjoits. Method of cojugate directios: A tutorial a a Published i SEP Report, 92, 253-365 (1996) Sergey Fomel 1 ABSTRACT This tutorial describes the classic method of cojugate

More information

6. Kalman filter implementation for linear algebraic equations. Karhunen-Loeve decomposition

6. Kalman filter implementation for linear algebraic equations. Karhunen-Loeve decomposition 6. Kalma filter implemetatio for liear algebraic equatios. Karhue-Loeve decompositio 6.1. Solvable liear algebraic systems. Probabilistic iterpretatio. Let A be a quadratic matrix (ot obligatory osigular.

More information

MATHEMATICAL SCIENCES PAPER-II

MATHEMATICAL SCIENCES PAPER-II MATHEMATICAL SCIENCES PAPER-II. Let {x } ad {y } be two sequeces of real umbers. Prove or disprove each of the statemets :. If {x y } coverges, ad if {y } is coverget, the {x } is coverget.. {x + y } coverges

More information

Multi parameter proximal point algorithms

Multi parameter proximal point algorithms Multi parameter proximal poit algorithms Ogaeditse A. Boikayo a,b,, Gheorghe Moroşau a a Departmet of Mathematics ad its Applicatios Cetral Europea Uiversity Nador u. 9, H-1051 Budapest, Hugary b Departmet

More information

Machine Learning Brett Bernstein

Machine Learning Brett Bernstein Machie Learig Brett Berstei Week 2 Lecture: Cocept Check Exercises Starred problems are optioal. Excess Risk Decompositio 1. Let X = Y = {1, 2,..., 10}, A = {1,..., 10, 11} ad suppose the data distributio

More information

1. By using truth tables prove that, for all statements P and Q, the statement

1. By using truth tables prove that, for all statements P and Q, the statement Author: Satiago Salazar Problems I: Mathematical Statemets ad Proofs. By usig truth tables prove that, for all statemets P ad Q, the statemet P Q ad its cotrapositive ot Q (ot P) are equivalet. I example.2.3

More information