MATH 5720: Gradient Methods Hung Phan, UMass Lowell October 4, 2018
|
|
- Beatrice Parker
- 5 years ago
- Views:
Transcription
1 MATH 5720: Gradien Mehods Hung Phan, UMass Lowell Ocober 4, 208 Descen Direcion Mehods Consider he problem min { f(x) x R n}. The general descen direcions mehod is x k+ = x k + k d k where x k is he curren ieraion; x k+ is he nex ieraion; d k is a (descen) direcion; k is he sepsize. Le f : R n R be coninuously differeniable. A vecor d is called a descen direcion of f a x if he direcional derivaive f (x; d) is negaive f f(x + d) f(x) (x; d) = lim = f(x) T d < 0. 0 Example. Le f(x, x 2 ) = x 2 + x2 2 x x 2 2x. Check if d = (2, ) is a descen direcion of f a x = (, ). Soluion. f(x, x 2 ) = (2x x 2 2, 2x 2 x ). So f(x) = (, ). So f (x; d) = f(x) T d = < 0. Thus, d is a descen direcion. Theorem 2. Suppose d is a descen direcion, hen here exiss δ > 0 such ha f(x + d) < f(x) for all (0, δ). Schemaic descen direcions mehod: Pick a saring poin x 0. For k = 0,, 2,... (a) Pick a descen direcion d k. (b) Find a sepsize k such ha f(x k + k d k ) < f(x k ). (c) Se x k+ = x k + k d k. (d) If sopping crieria is saisfied, hen STOP, oherwise, go o (a). For descen direcions, here are many choices. For sepsize, he hree common choices are (i) consan sepsize: k = for all k. (ii) exac line search: k is a minimizer of f along he ray f(x k + d k ). (iii) backracking: Take s > 0, α, β (0, ). Firs, se k = s, hen while f(x k ) f(x k + k d k ) k < α f(x k ) T d k se k β k. The las opion finds a sepsize ha mees he sufficien decrease condiion f(x k ) f(x k + k d k ) k α f(x k ) T d k.
2 Example 3 (exac line search). Le f(x, x 2 ) = x 2 + x2 2 x x 2 2x. Perform an exac line search in direcion d. Soluion. To perform a line search on d, firs, g() = f(x + d) = f( + 2, + ) = ( + 2) 2 + ( + ) 2 ( + 2)( + ) 2( + 2) = 3 2 So we find he minimizer of g() over 0. Take g () = 6 = 0, so = 6. Since g () = 6 > 0 and g is quadraic, = 6 is he global minimizer. So, he exac line search on d yields = argmin f(x + d) = 6. 0 And we can check ha f(x) = f(, ) = and f(x + d) == f( 8 6, 7 6 ) = 3 2. Example 4 (exac line search for quadraic funcion). Le f(x) = x T Ax + 2b T x + c where A R n n is posiive definie, b R n and c R. Le x R n and d R n be a descen direcion of f a x. We will find an explici formula for he sepsize generaed by exac line search, i.e., a soluion of min f(x + d). 0 We have g() = f(x + d) = (x + d) T A(x + d) + 2b T (x + d) + c = (d T Ad) 2 + 2(d T Ax + d T b) + x T Ax + 2b T x + c = (d T Ad) 2 + 2(d T Ax + d T b) + f(x). Since g () = 2(d T Ad) + 2d T (Ax + b) and f(x) = 2(Ax + b), i follows ha g ( ) = 0 if and only if = dt f(x) 2d T Ad. Since d is a descen direcion, d T f(x) < 0, hence > 0. So is he desired soluion. Example 5 (backracking). Le f(x, x 2 ) = x 2 + x2 2 x x 2 2x and x = (, ). backracking in direcion d = (2, ) wih s =, α = 0.7, β = 0.5. Soluion. Firs, α f(x) T d = 0.7. We have he able So, backracking procedure produces = f(x) f(x+d) Perform Lemma 6 (validiy of sufficien decrease condiion). Suppose f : R n R is coninuously differeniable and d is a descen direcion a x, and α (0, ). Then here exiss δ > 0 such ha f(x) f(x + d) α f(x) T d for all (0, δ). 2
3 f(x + d) f(x) Proof. Since f is differeniable, we have lim = f (x; d) = f(x) T d < 0. 0 f(x + d) f(x) Thus, here exiss δ > 0 such ha < α f(x) T d for all (0, δ). 2 Gradien Mehod In he gradien mehod, he descen direcion is chosen o be he opposie of he gradien a he curren ieraion d k = f(x k ). Lemma 7. Le f be coninuously differeniable, and le x be a nonsaionary poin. opimal soluion of Then an Proof. For all d wih d =, min d { f (x; d) d R n, d = } is d = f(x) f(x). f (x; d) = f(x) T d f(x) d = f(x). Thus, f(x) is a lower bound for he problem. Now using d = f(x) f(x), we have f ( ) ( ) x, f(x) f(x) = f(x) T f(x) f(x) = f(x). So he lower bound is aained, which means f(x) f(x) is an opimal soluion (minimizer). The gradien mehod Inpu: ε > 0 olerance parameer and pick x 0 R n arbirarily. For k = 0,, 2,... (a) Pick a sepsize k by a line search on he funcion g() = f(x k f(x k )). (b) Se x k+ = x k f(x k ). (c) If f(x k+ ) ε hen STOP and x k+ is he oupu, oherwise repea. Example 8 (gradien mehod wih exac line search for quadraic funcion). Consider min f(x) = x R xt Ax + 2b T x n where A R n n is posiive definie and b R n. The gradien mehod wih sepsize chosen by exac line search generaes x k+ := x k k f(x k ) where k = f(x k ) 2 2 f(x k ) T A f(x k ). Proposiion 9. Le {x k } be a sequence generaed by he gradien mehod wih exac line search for solving a problem of minimizing a coninuously differeniable funcion f. Then for any k = 0,, 2,..., (x k+2 x k+ ) T (x k+ x k ) = 0. 3
4 Proof. By definiion, x k+ x k = k f(x k ) and x k+2 x k+ = k+ f(x k+ ). So, i suffices o prove ha f(x k ) T f(x k+ ) = 0. Since k = argmin g() = f(x k f(x k )), 0 and k = 0 is no he soluion, ha implies g ( k ) = 0. Thus, f(x k ) T f(x k k f(x k )) = 0 f(x k ) T f(x k+ ) = 0. 3 The Ferma-Weber Problem The problem is se up as follows: given a se of m poins A := {a, a 2,..., a m } R n (also called anchor poins) and m weighs ω,..., ω m > 0, find a poin x R n ha minimize he weighed disance of x o each of he poins a i : { min f(x) x R n } x a i. Noe ha f is no differeniable a he anchor poins a i. This is an insance of a faciliy locaion problem. One popular approach for solving he problem was inroduced by Weiszfeld in 937: consider he firs order opimaliy condiion f(x) = 0 where we implicily assume x is no an anchor poin. Then he equaion can be wrien explicily as x a ( m i x a i = 0 ω ) i a i x = x a i x a i, which is he same as x = x a i So he opimaliy condiion becomes a fixed poin problem a i x a i. Finding an x such ha x = T (x) where T (x) = x a i a i x a i. A naural approach is o use he ieraions x k+ = T (x k ) which is he Weiszfeld s mehod. This mehod defined only when all x k s are differen from a i s. Surprisingly, his mehod is basically a gradien mehod. Indeed, ω x k+ = i a i m x k a i x k a i x k a i = x k x k a i = x k x a i x k a i 4 f(x k ).
5 So Weiszfeld s mehod is he gradien mehod wih special sepsize k = x k a i Several quesions hen arise: Is he mehod well-defined? Does he value of objecive funcion decrease? Does he sequence {x k } k N converge o a global soluion? In he following, we will answer par of hese quesions. Firs, define he auxiliary funcion h(y, x) := Lemma 0. For any x R n A, one has y a i 2 x a i, y Rn, x R n A. T (x) = argmin y R n h(y, x) ( m ) ω Proof. The funcion h(, x) is quadraic whose associaed marix is i x a i I, which is posiive definie. Therefore, he unique global minimizer, denoed by y, is he unique saionary poin of h(, x), i.e., y h(y, x) = 0 2 w i y a i x a i = 0 y = T (x). Lemma 0 basically shows ha he Weiszfeld s mehod can be wrien as Lemma. If x R n A, hen (i) h(x, x) = f(x). (ii) h(y, x) 2f(y) f(x) for all y R n. x k+ = argmin x R n h(x, x k ). (iii) f(t (x)) f(x) and f(t (x)) = f(x) if and only if x = T (x). (iv) x = T (x) if and only if f(x) = 0.. Proof. (i): obvious. (ii): For all a 0 and b > 0, he following always holds: a2 b The conclusion hen follows easily. (iii): I follows from (ii) ha 2a b. () 2f(T (x)) f(x) h(t (x), x) h(x, x) = f(x). Thus, f(x) f(t (x)). To complee his par: if T (x) = x hen clearly f(t (x)) = f(x). Now if f(t x) = f(x), hen from () we have h(t (x), x) = h(x, x). Since h(, x) has a unique minimizer T (x), we mus have x = T (x). (iv): follows from simple algebraic manipulaion. Lemma 2. Le {x k } k N be he sequence generaed by Weiszfeld s mehod, where we assume ha x k A for all k N. Then {f(x k )} k N is nonincreasing and f(x k ) = f(x k+ ) if and only if f(x k ) = 0. 5
6 Proof. The conclusion is immediae from Lemma. We have showed ha {f(x k )} k N is sricly decreasing as long as we are no suck a a saionary poin. The underlying assumpion ha x k A is problemaic since i canno be verified easily. One approach is o choose he saring poin x 0 such ha So, we have he following resul. f(x 0 ) < min a i A {f(a i)}. Theorem 3. Le {x k } k N be he sequence generaed by Weiszfeld s mehod and assume ha f(x 0 ) < min ai A{f(a i )}. Then all limi poins of {x k } k N are saionary poins of f. I can be showed ha he saionary poins of he Ferma-Weber problem are global opimal soluions. In fac, i is also possible o show ha he enire sequence converges o a global opimal soluion. References [] A. Beck, Inroducion o Nonlinear Opimizaion: Theory, Algorihms, and Applicaions wih MATLAB, MOS-SIAM Series on Opimizaion (204). 6
Course Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course Noes for EE7C Spring 018: Convex Opimizaion and Approximaion Insrucor: Moriz Hard Email: hard+ee7c@berkeley.edu Graduae Insrucor: Max Simchowiz Email: msimchow+ee7c@berkeley.edu Ocober 15, 018 3
More informationOptimality Conditions for Unconstrained Problems
62 CHAPTER 6 Opimaliy Condiions for Unconsrained Problems 1 Unconsrained Opimizaion 11 Exisence Consider he problem of minimizing he funcion f : R n R where f is coninuous on all of R n : P min f(x) x
More informationA Primal-Dual Type Algorithm with the O(1/t) Convergence Rate for Large Scale Constrained Convex Programs
PROC. IEEE CONFERENCE ON DECISION AND CONTROL, 06 A Primal-Dual Type Algorihm wih he O(/) Convergence Rae for Large Scale Consrained Convex Programs Hao Yu and Michael J. Neely Absrac This paper considers
More informationMATH 128A, SUMMER 2009, FINAL EXAM SOLUTION
MATH 28A, SUMME 2009, FINAL EXAM SOLUTION BENJAMIN JOHNSON () (8 poins) [Lagrange Inerpolaion] (a) (4 poins) Le f be a funcion defined a some real numbers x 0,..., x n. Give a defining equaion for he Lagrange
More informationChapter 3 Boundary Value Problem
Chaper 3 Boundary Value Problem A boundary value problem (BVP) is a problem, ypically an ODE or a PDE, which has values assigned on he physical boundary of he domain in which he problem is specified. Le
More informationAn Introduction to Malliavin calculus and its applications
An Inroducion o Malliavin calculus and is applicaions Lecure 5: Smoohness of he densiy and Hörmander s heorem David Nualar Deparmen of Mahemaics Kansas Universiy Universiy of Wyoming Summer School 214
More informationFinish reading Chapter 2 of Spivak, rereading earlier sections as necessary. handout and fill in some missing details!
MAT 257, Handou 6: Ocober 7-2, 20. I. Assignmen. Finish reading Chaper 2 of Spiva, rereading earlier secions as necessary. handou and fill in some missing deails! II. Higher derivaives. Also, read his
More informationIMPLICIT AND INVERSE FUNCTION THEOREMS PAUL SCHRIMPF 1 OCTOBER 25, 2013
IMPLICI AND INVERSE FUNCION HEOREMS PAUL SCHRIMPF 1 OCOBER 25, 213 UNIVERSIY OF BRIISH COLUMBIA ECONOMICS 526 We have exensively sudied how o solve sysems of linear equaions. We know how o check wheher
More informationEssential Microeconomics : OPTIMAL CONTROL 1. Consider the following class of optimization problems
Essenial Microeconomics -- 6.5: OPIMAL CONROL Consider he following class of opimizaion problems Max{ U( k, x) + U+ ( k+ ) k+ k F( k, x)}. { x, k+ } = In he language of conrol heory, he vecor k is he vecor
More informationMATH 4330/5330, Fourier Analysis Section 6, Proof of Fourier s Theorem for Pointwise Convergence
MATH 433/533, Fourier Analysis Secion 6, Proof of Fourier s Theorem for Poinwise Convergence Firs, some commens abou inegraing periodic funcions. If g is a periodic funcion, g(x + ) g(x) for all real x,
More informationThe Asymptotic Behavior of Nonoscillatory Solutions of Some Nonlinear Dynamic Equations on Time Scales
Advances in Dynamical Sysems and Applicaions. ISSN 0973-5321 Volume 1 Number 1 (2006, pp. 103 112 c Research India Publicaions hp://www.ripublicaion.com/adsa.hm The Asympoic Behavior of Nonoscillaory Soluions
More informationChapter 2. First Order Scalar Equations
Chaper. Firs Order Scalar Equaions We sar our sudy of differenial equaions in he same way he pioneers in his field did. We show paricular echniques o solve paricular ypes of firs order differenial equaions.
More information15. Vector Valued Functions
1. Vecor Valued Funcions Up o his poin, we have presened vecors wih consan componens, for example, 1, and,,4. However, we can allow he componens of a vecor o be funcions of a common variable. For example,
More informationConvergence of the Neumann series in higher norms
Convergence of he Neumann series in higher norms Charles L. Epsein Deparmen of Mahemaics, Universiy of Pennsylvania Version 1.0 Augus 1, 003 Absrac Naural condiions on an operaor A are given so ha he Neumann
More information12: AUTOREGRESSIVE AND MOVING AVERAGE PROCESSES IN DISCRETE TIME. Σ j =
1: AUTOREGRESSIVE AND MOVING AVERAGE PROCESSES IN DISCRETE TIME Moving Averages Recall ha a whie noise process is a series { } = having variance σ. The whie noise process has specral densiy f (λ) = of
More informationCash Flow Valuation Mode Lin Discrete Time
IOSR Journal of Mahemaics (IOSR-JM) e-issn: 2278-5728,p-ISSN: 2319-765X, 6, Issue 6 (May. - Jun. 2013), PP 35-41 Cash Flow Valuaion Mode Lin Discree Time Olayiwola. M. A. and Oni, N. O. Deparmen of Mahemaics
More informationLecture 9: September 25
0-725: Opimizaion Fall 202 Lecure 9: Sepember 25 Lecurer: Geoff Gordon/Ryan Tibshirani Scribes: Xuezhi Wang, Subhodeep Moira, Abhimanu Kumar Noe: LaTeX emplae couresy of UC Berkeley EECS dep. Disclaimer:
More informationLecture 20: Riccati Equations and Least Squares Feedback Control
34-5 LINEAR SYSTEMS Lecure : Riccai Equaions and Leas Squares Feedback Conrol 5.6.4 Sae Feedback via Riccai Equaions A recursive approach in generaing he marix-valued funcion W ( ) equaion for i for he
More informationSupplement for Stochastic Convex Optimization: Faster Local Growth Implies Faster Global Convergence
Supplemen for Sochasic Convex Opimizaion: Faser Local Growh Implies Faser Global Convergence Yi Xu Qihang Lin ianbao Yang Proof of heorem heorem Suppose Assumpion holds and F (w) obeys he LGC (6) Given
More informationSome Basic Information about M-S-D Systems
Some Basic Informaion abou M-S-D Sysems 1 Inroducion We wan o give some summary of he facs concerning unforced (homogeneous) and forced (non-homogeneous) models for linear oscillaors governed by second-order,
More informationINEXACT CUTS FOR DETERMINISTIC AND STOCHASTIC DUAL DYNAMIC PROGRAMMING APPLIED TO CONVEX NONLINEAR OPTIMIZATION PROBLEMS
INEXACT CUTS FOR DETERMINISTIC AND STOCHASTIC DUAL DYNAMIC PROGRAMMING APPLIED TO CONVEX NONLINEAR OPTIMIZATION PROBLEMS Vincen Guigues School of Applied Mahemaics, FGV Praia de Boafogo, Rio de Janeiro,
More information10. State Space Methods
. Sae Space Mehods. Inroducion Sae space modelling was briefly inroduced in chaper. Here more coverage is provided of sae space mehods before some of heir uses in conrol sysem design are covered in he
More informationOn-line Adaptive Optimal Timing Control of Switched Systems
On-line Adapive Opimal Timing Conrol of Swiched Sysems X.C. Ding, Y. Wardi and M. Egersed Absrac In his paper we consider he problem of opimizing over he swiching imes for a muli-modal dynamic sysem when
More informationd 1 = c 1 b 2 - b 1 c 2 d 2 = c 1 b 3 - b 1 c 3
and d = c b - b c c d = c b - b c c This process is coninued unil he nh row has been compleed. The complee array of coefficiens is riangular. Noe ha in developing he array an enire row may be divided or
More informationELE 538B: Large-Scale Optimization for Data Science. Quasi-Newton methods. Yuxin Chen Princeton University, Spring 2018
ELE 538B: Large-Scale Opimizaion for Daa Science Quasi-Newon mehods Yuxin Chen Princeon Universiy, Spring 208 00 op ff(x (x)(k)) f p 2 L µ f 05 k f (xk ) k f (xk ) =) f op ieraions converges in only 5
More informationMatrix Versions of Some Refinements of the Arithmetic-Geometric Mean Inequality
Marix Versions of Some Refinemens of he Arihmeic-Geomeric Mean Inequaliy Bao Qi Feng and Andrew Tonge Absrac. We esablish marix versions of refinemens due o Alzer ], Carwrigh and Field 4], and Mercer 5]
More information1 Solutions to selected problems
1 Soluions o seleced problems 1. Le A B R n. Show ha in A in B bu in general bd A bd B. Soluion. Le x in A. Then here is ɛ > 0 such ha B ɛ (x) A B. This shows x in B. If A = [0, 1] and B = [0, 2], hen
More informationQuestion 1: Question 2: Topology Exercise Sheet 3
Topology Exercise Shee 3 Prof. Dr. Alessandro Siso Due o 14 March Quesions 1 and 6 are more concepual and should have prioriy. Quesions 4 and 5 admi a relaively shor soluion. Quesion 7 is harder, and you
More informationSection 3.5 Nonhomogeneous Equations; Method of Undetermined Coefficients
Secion 3.5 Nonhomogeneous Equaions; Mehod of Undeermined Coefficiens Key Terms/Ideas: Linear Differenial operaor Nonlinear operaor Second order homogeneous DE Second order nonhomogeneous DE Soluion o homogeneous
More informationChapter 8 The Complete Response of RL and RC Circuits
Chaper 8 The Complee Response of RL and RC Circuis Seoul Naional Universiy Deparmen of Elecrical and Compuer Engineering Wha is Firs Order Circuis? Circuis ha conain only one inducor or only one capacior
More informationArticle from. Predictive Analytics and Futurism. July 2016 Issue 13
Aricle from Predicive Analyics and Fuurism July 6 Issue An Inroducion o Incremenal Learning By Qiang Wu and Dave Snell Machine learning provides useful ools for predicive analyics The ypical machine learning
More informationOrdinary Differential Equations
Ordinary Differenial Equaions 5. Examples of linear differenial equaions and heir applicaions We consider some examples of sysems of linear differenial equaions wih consan coefficiens y = a y +... + a
More informationEXERCISES FOR SECTION 1.5
1.5 Exisence and Uniqueness of Soluions 43 20. 1 v c 21. 1 v c 1 2 4 6 8 10 1 2 2 4 6 8 10 Graph of approximae soluion obained using Euler s mehod wih = 0.1. Graph of approximae soluion obained using Euler
More informationINTRODUCTION TO MACHINE LEARNING 3RD EDITION
ETHEM ALPAYDIN The MIT Press, 2014 Lecure Slides for INTRODUCTION TO MACHINE LEARNING 3RD EDITION alpaydin@boun.edu.r hp://www.cmpe.boun.edu.r/~ehem/i2ml3e CHAPTER 2: SUPERVISED LEARNING Learning a Class
More informationStability and Bifurcation in a Neural Network Model with Two Delays
Inernaional Mahemaical Forum, Vol. 6, 11, no. 35, 175-1731 Sabiliy and Bifurcaion in a Neural Nework Model wih Two Delays GuangPing Hu and XiaoLing Li School of Mahemaics and Physics, Nanjing Universiy
More informationOscillation of an Euler Cauchy Dynamic Equation S. Huff, G. Olumolode, N. Pennington, and A. Peterson
PROCEEDINGS OF THE FOURTH INTERNATIONAL CONFERENCE ON DYNAMICAL SYSTEMS AND DIFFERENTIAL EQUATIONS May 4 7, 00, Wilmingon, NC, USA pp 0 Oscillaion of an Euler Cauchy Dynamic Equaion S Huff, G Olumolode,
More informationHamilton- J acobi Equation: Weak S olution We continue the study of the Hamilton-Jacobi equation:
M ah 5 7 Fall 9 L ecure O c. 4, 9 ) Hamilon- J acobi Equaion: Weak S oluion We coninue he sudy of he Hamilon-Jacobi equaion: We have shown ha u + H D u) = R n, ) ; u = g R n { = }. ). In general we canno
More informationLet us start with a two dimensional case. We consider a vector ( x,
Roaion marices We consider now roaion marices in wo and hree dimensions. We sar wih wo dimensions since wo dimensions are easier han hree o undersand, and one dimension is a lile oo simple. However, our
More informationInventory Analysis and Management. Multi-Period Stochastic Models: Optimality of (s, S) Policy for K-Convex Objective Functions
Muli-Period Sochasic Models: Opimali of (s, S) Polic for -Convex Objecive Funcions Consider a seing similar o he N-sage newsvendor problem excep ha now here is a fixed re-ordering cos (> 0) for each (re-)order.
More informationA Decentralized Second-Order Method with Exact Linear Convergence Rate for Consensus Optimization
1 A Decenralized Second-Order Mehod wih Exac Linear Convergence Rae for Consensus Opimizaion Aryan Mokhari, Wei Shi, Qing Ling, and Alejandro Ribeiro Absrac This paper considers decenralized consensus
More information4. Advanced Stability Theory
Applied Nonlinear Conrol Nguyen an ien - 4 4 Advanced Sabiliy heory he objecive of his chaper is o presen sabiliy analysis for non-auonomous sysems 41 Conceps of Sabiliy for Non-Auonomous Sysems Equilibrium
More informationVanishing Viscosity Method. There are another instructive and perhaps more natural discontinuous solutions of the conservation law
Vanishing Viscosiy Mehod. There are anoher insrucive and perhaps more naural disconinuous soluions of he conservaion law (1 u +(q(u x 0, he so called vanishing viscosiy mehod. This mehod consiss in viewing
More informationMath Final Exam Solutions
Mah 246 - Final Exam Soluions Friday, July h, 204 () Find explici soluions and give he inerval of definiion o he following iniial value problems (a) ( + 2 )y + 2y = e, y(0) = 0 Soluion: In normal form,
More informationPOSITIVE SOLUTIONS OF NEUTRAL DELAY DIFFERENTIAL EQUATION
Novi Sad J. Mah. Vol. 32, No. 2, 2002, 95-108 95 POSITIVE SOLUTIONS OF NEUTRAL DELAY DIFFERENTIAL EQUATION Hajnalka Péics 1, János Karsai 2 Absrac. We consider he scalar nonauonomous neural delay differenial
More informationSOLUTIONS TO ECE 3084
SOLUTIONS TO ECE 384 PROBLEM 2.. For each sysem below, specify wheher or no i is: (i) memoryless; (ii) causal; (iii) inverible; (iv) linear; (v) ime invarian; Explain your reasoning. If he propery is no
More informationCorrespondence should be addressed to Nguyen Buong,
Hindawi Publishing Corporaion Fixed Poin Theory and Applicaions Volume 011, Aricle ID 76859, 10 pages doi:101155/011/76859 Research Aricle An Implici Ieraion Mehod for Variaional Inequaliies over he Se
More information(1) (2) Differentiation of (1) and then substitution of (3) leads to. Therefore, we will simply consider the second-order linear system given by (4)
Phase Plane Analysis of Linear Sysems Adaped from Applied Nonlinear Conrol by Sloine and Li The general form of a linear second-order sysem is a c b d From and b bc d a Differeniaion of and hen subsiuion
More informationIntermediate Differential Equations Review and Basic Ideas
Inermediae Differenial Equaions Review and Basic Ideas John A. Burns Cener for Opimal Design And Conrol Inerdisciplinary Cener forappliedmahemaics Virginia Polyechnic Insiue and Sae Universiy Blacksburg,
More information1 1 + x 2 dx. tan 1 (2) = ] ] x 3. Solution: Recall that the given integral is improper because. x 3. 1 x 3. dx = lim dx.
. Use Simpson s rule wih n 4 o esimae an () +. Soluion: Since we are using 4 seps, 4 Thus we have [ ( ) f() + 4f + f() + 4f 3 [ + 4 4 6 5 + + 4 4 3 + ] 5 [ + 6 6 5 + + 6 3 + ]. 5. Our funcion is f() +.
More informationProjection-Based Optimal Mode Scheduling
Projecion-Based Opimal Mode Scheduling T. M. Caldwell and T. D. Murphey Absrac This paper develops an ieraive opimizaion echnique ha can be applied o mode scheduling. The algorihm provides boh a mode schedule
More informationLecture 2 October ε-approximation of 2-player zero-sum games
Opimizaion II Winer 009/10 Lecurer: Khaled Elbassioni Lecure Ocober 19 1 ε-approximaion of -player zero-sum games In his lecure we give a randomized ficiious play algorihm for obaining an approximae soluion
More informationSTABILITY OF PEXIDERIZED QUADRATIC FUNCTIONAL EQUATION IN NON-ARCHIMEDEAN FUZZY NORMED SPASES
Novi Sad J. Mah. Vol. 46, No. 1, 2016, 15-25 STABILITY OF PEXIDERIZED QUADRATIC FUNCTIONAL EQUATION IN NON-ARCHIMEDEAN FUZZY NORMED SPASES N. Eghbali 1 Absrac. We deermine some sabiliy resuls concerning
More informationHamilton- J acobi Equation: Explicit Formulas In this lecture we try to apply the method of characteristics to the Hamilton-Jacobi equation: u t
M ah 5 2 7 Fall 2 0 0 9 L ecure 1 0 O c. 7, 2 0 0 9 Hamilon- J acobi Equaion: Explici Formulas In his lecure we ry o apply he mehod of characerisics o he Hamilon-Jacobi equaion: u + H D u, x = 0 in R n
More informationTracking Adversarial Targets
A. Proofs Proof of Lemma 3. Consider he Bellman equaion λ + V π,l x, a lx, a + V π,l Ax + Ba, πax + Ba. We prove he lemma by showing ha he given quadraic form is he unique soluion of he Bellman equaion.
More informationHeat kernel and Harnack inequality on Riemannian manifolds
Hea kernel and Harnack inequaliy on Riemannian manifolds Alexander Grigor yan UHK 11/02/2014 onens 1 Laplace operaor and hea kernel 1 2 Uniform Faber-Krahn inequaliy 3 3 Gaussian upper bounds 4 4 ean-value
More informationNotes for Lecture 17-18
U.C. Berkeley CS278: Compuaional Complexiy Handou N7-8 Professor Luca Trevisan April 3-8, 2008 Noes for Lecure 7-8 In hese wo lecures we prove he firs half of he PCP Theorem, he Amplificaion Lemma, up
More informationAryan Mokhtari, Wei Shi, Qing Ling, and Alejandro Ribeiro. cost function n
IEEE TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING OVER NETWORKS, VOL. 2, NO. 4, DECEMBER 2016 507 A Decenralized Second-Order Mehod wih Exac Linear Convergence Rae for Consensus Opimizaion Aryan Mokhari,
More information1. An introduction to dynamic optimization -- Optimal Control and Dynamic Programming AGEC
This documen was generaed a :45 PM 8/8/04 Copyrigh 04 Richard T. Woodward. An inroducion o dynamic opimizaion -- Opimal Conrol and Dynamic Programming AGEC 637-04 I. Overview of opimizaion Opimizaion is
More informationCHAPTER 12 DIRECT CURRENT CIRCUITS
CHAPTER 12 DIRECT CURRENT CIUITS DIRECT CURRENT CIUITS 257 12.1 RESISTORS IN SERIES AND IN PARALLEL When wo resisors are conneced ogeher as shown in Figure 12.1 we said ha hey are conneced in series. As
More informationarxiv: v1 [math.ca] 15 Nov 2016
arxiv:6.599v [mah.ca] 5 Nov 26 Counerexamples on Jumarie s hree basic fracional calculus formulae for non-differeniable coninuous funcions Cheng-shi Liu Deparmen of Mahemaics Norheas Peroleum Universiy
More information!!"#"$%&#'()!"#&'(*%)+,&',-)./0)1-*23)
"#"$%&#'()"#&'(*%)+,&',-)./)1-*) #$%&'()*+,&',-.%,/)*+,-&1*#$)()5*6$+$%*,7&*-'-&1*(,-&*6&,7.$%$+*&%'(*8$&',-,%'-&1*(,-&*6&,79*(&,%: ;..,*&1$&$.$%&'()*1$$.,'&',-9*(&,%)?%*,('&5
More informationHomework 10 (Stats 620, Winter 2017) Due Tuesday April 18, in class Questions are derived from problems in Stochastic Processes by S. Ross.
Homework (Sas 6, Winer 7 Due Tuesday April 8, in class Quesions are derived from problems in Sochasic Processes by S. Ross.. A sochasic process {X(, } is said o be saionary if X(,..., X( n has he same
More informationEXISTENCE OF NON-OSCILLATORY SOLUTIONS TO FIRST-ORDER NEUTRAL DIFFERENTIAL EQUATIONS
Elecronic Journal of Differenial Equaions, Vol. 206 (206, No. 39, pp.. ISSN: 072-669. URL: hp://ejde.mah.xsae.edu or hp://ejde.mah.un.edu fp ejde.mah.xsae.edu EXISTENCE OF NON-OSCILLATORY SOLUTIONS TO
More informationAnnouncements: Warm-up Exercise:
Fri Apr 13 7.1 Sysems of differenial equaions - o model muli-componen sysems via comparmenal analysis hp//en.wikipedia.org/wiki/muli-comparmen_model Announcemens Warm-up Exercise Here's a relaively simple
More informationWEEK-3 Recitation PHYS 131. of the projectile s velocity remains constant throughout the motion, since the acceleration a x
WEEK-3 Reciaion PHYS 131 Ch. 3: FOC 1, 3, 4, 6, 14. Problems 9, 37, 41 & 71 and Ch. 4: FOC 1, 3, 5, 8. Problems 3, 5 & 16. Feb 8, 018 Ch. 3: FOC 1, 3, 4, 6, 14. 1. (a) The horizonal componen of he projecile
More informationSimulation-Solving Dynamic Models ABE 5646 Week 2, Spring 2010
Simulaion-Solving Dynamic Models ABE 5646 Week 2, Spring 2010 Week Descripion Reading Maerial 2 Compuer Simulaion of Dynamic Models Finie Difference, coninuous saes, discree ime Simple Mehods Euler Trapezoid
More informationA Forward-Backward Splitting Method with Component-wise Lazy Evaluation for Online Structured Convex Optimization
A Forward-Backward Spliing Mehod wih Componen-wise Lazy Evaluaion for Online Srucured Convex Opimizaion Yukihiro Togari and Nobuo Yamashia March 28, 2016 Absrac: We consider large-scale opimizaion problems
More informationA Shooting Method for A Node Generation Algorithm
A Shooing Mehod for A Node Generaion Algorihm Hiroaki Nishikawa W.M.Keck Foundaion Laboraory for Compuaional Fluid Dynamics Deparmen of Aerospace Engineering, Universiy of Michigan, Ann Arbor, Michigan
More informationApproximation Algorithms for Unique Games via Orthogonal Separators
Approximaion Algorihms for Unique Games via Orhogonal Separaors Lecure noes by Konsanin Makarychev. Lecure noes are based on he papers [CMM06a, CMM06b, LM4]. Unique Games In hese lecure noes, we define
More informationModal identification of structures from roving input data by means of maximum likelihood estimation of the state space model
Modal idenificaion of srucures from roving inpu daa by means of maximum likelihood esimaion of he sae space model J. Cara, J. Juan, E. Alarcón Absrac The usual way o perform a forced vibraion es is o fix
More informationVariational Iteration Method for Solving System of Fractional Order Ordinary Differential Equations
IOSR Journal of Mahemaics (IOSR-JM) e-issn: 2278-5728, p-issn: 2319-765X. Volume 1, Issue 6 Ver. II (Nov - Dec. 214), PP 48-54 Variaional Ieraion Mehod for Solving Sysem of Fracional Order Ordinary Differenial
More informationAnn. Funct. Anal. 2 (2011), no. 2, A nnals of F unctional A nalysis ISSN: (electronic) URL:
Ann. Func. Anal. 2 2011, no. 2, 34 41 A nnals of F uncional A nalysis ISSN: 2008-8752 elecronic URL: www.emis.de/journals/afa/ CLASSIFICAION OF POSIIVE SOLUIONS OF NONLINEAR SYSEMS OF VOLERRA INEGRAL EQUAIONS
More informationThe Rosenblatt s LMS algorithm for Perceptron (1958) is built around a linear neuron (a neuron with a linear
In The name of God Lecure4: Percepron and AALIE r. Majid MjidGhoshunih Inroducion The Rosenbla s LMS algorihm for Percepron 958 is buil around a linear neuron a neuron ih a linear acivaion funcion. Hoever,
More informationdy dx = xey (a) y(0) = 2 (b) y(1) = 2.5 SOLUTION: See next page
Assignmen 1 MATH 2270 SOLUTION Please wrie ou complee soluions for each of he following 6 problems (one more will sill be added). You may, of course, consul wih your classmaes, he exbook or oher resources,
More informationSolutions to Assignment 1
MA 2326 Differenial Equaions Insrucor: Peronela Radu Friday, February 8, 203 Soluions o Assignmen. Find he general soluions of he following ODEs: (a) 2 x = an x Soluion: I is a separable equaion as we
More informationUndetermined coefficients for local fractional differential equations
Available online a www.isr-publicaions.com/jmcs J. Mah. Compuer Sci. 16 (2016), 140 146 Research Aricle Undeermined coefficiens for local fracional differenial equaions Roshdi Khalil a,, Mohammed Al Horani
More informationExistence Theory of Second Order Random Differential Equations
Global Journal of Mahemaical Sciences: Theory and Pracical. ISSN 974-32 Volume 4, Number 3 (22), pp. 33-3 Inernaional Research Publicaion House hp://www.irphouse.com Exisence Theory of Second Order Random
More informationDistance Between Two Ellipses in 3D
Disance Beween Two Ellipses in 3D David Eberly Magic Sofware 6006 Meadow Run Cour Chapel Hill, NC 27516 eberly@magic-sofware.com 1 Inroducion An ellipse in 3D is represened by a cener C, uni lengh axes
More informationOnline Convex Optimization Example And Follow-The-Leader
CSE599s, Spring 2014, Online Learning Lecure 2-04/03/2014 Online Convex Opimizaion Example And Follow-The-Leader Lecurer: Brendan McMahan Scribe: Sephen Joe Jonany 1 Review of Online Convex Opimizaion
More informationDimitri Solomatine. D.P. Solomatine. Data-driven modelling (part 2). 2
Daa-driven modelling. Par. Daa-driven Arificial di Neural modelling. Newors Par Dimiri Solomaine Arificial neural newors D.P. Solomaine. Daa-driven modelling par. 1 Arificial neural newors ANN: main pes
More information6.2 Transforms of Derivatives and Integrals.
SEC. 6.2 Transforms of Derivaives and Inegrals. ODEs 2 3 33 39 23. Change of scale. If l( f ()) F(s) and c is any 33 45 APPLICATION OF s-shifting posiive consan, show ha l( f (c)) F(s>c)>c (Hin: In Probs.
More informationODEs II, Lecture 1: Homogeneous Linear Systems - I. Mike Raugh 1. March 8, 2004
ODEs II, Lecure : Homogeneous Linear Sysems - I Mike Raugh March 8, 4 Inroducion. In he firs lecure we discussed a sysem of linear ODEs for modeling he excreion of lead from he human body, saw how o ransform
More informationOptimality and complexity for constrained optimization problems with nonconvex regularization
Opimaliy and complexiy for consrained opimizaion problems wih nonconvex regularizaion Wei Bian Deparmen of Mahemaics, Harbin Insiue of Technology, Harbin, China, bianweilvse520@163.com Xiaojun Chen Deparmen
More informationExistence of multiple positive periodic solutions for functional differential equations
J. Mah. Anal. Appl. 325 (27) 1378 1389 www.elsevier.com/locae/jmaa Exisence of muliple posiive periodic soluions for funcional differenial equaions Zhijun Zeng a,b,,libi a, Meng Fan a a School of Mahemaics
More informationApplication of a Stochastic-Fuzzy Approach to Modeling Optimal Discrete Time Dynamical Systems by Using Large Scale Data Processing
Applicaion of a Sochasic-Fuzzy Approach o Modeling Opimal Discree Time Dynamical Sysems by Using Large Scale Daa Processing AA WALASZE-BABISZEWSA Deparmen of Compuer Engineering Opole Universiy of Technology
More informationKINEMATICS IN ONE DIMENSION
KINEMATICS IN ONE DIMENSION PREVIEW Kinemaics is he sudy of how hings move how far (disance and displacemen), how fas (speed and velociy), and how fas ha how fas changes (acceleraion). We say ha an objec
More informationNetwork Newton Distributed Optimization Methods
Nework Newon Disribued Opimizaion Mehods Aryan Mokhari, Qing Ling, and Alejandro Ribeiro Absrac We sudy he problem of minimizing a sum of convex objecive funcions where he componens of he objecive are
More informationMODULE 3 FUNCTION OF A RANDOM VARIABLE AND ITS DISTRIBUTION LECTURES PROBABILITY DISTRIBUTION OF A FUNCTION OF A RANDOM VARIABLE
Topics MODULE 3 FUNCTION OF A RANDOM VARIABLE AND ITS DISTRIBUTION LECTURES 2-6 3. FUNCTION OF A RANDOM VARIABLE 3.2 PROBABILITY DISTRIBUTION OF A FUNCTION OF A RANDOM VARIABLE 3.3 EXPECTATION AND MOMENTS
More information6. Stochastic calculus with jump processes
A) Trading sraegies (1/3) Marke wih d asses S = (S 1,, S d ) A rading sraegy can be modelled wih a vecor φ describing he quaniies invesed in each asse a each insan : φ = (φ 1,, φ d ) The value a of a porfolio
More informationPROOF FOR A CASE WHERE DISCOUNTING ADVANCES THE DOOMSDAY. T. C. Koopmans
PROOF FOR A CASE WHERE DISCOUNTING ADVANCES THE DOOMSDAY T. C. Koopmans January 1974 WP-74-6 Working Papers are no inended for disribuion ouside of IIASA, and are solely for discussion and informaion purposes.
More informationLecture 2-1 Kinematics in One Dimension Displacement, Velocity and Acceleration Everything in the world is moving. Nothing stays still.
Lecure - Kinemaics in One Dimension Displacemen, Velociy and Acceleraion Everyhing in he world is moving. Nohing says sill. Moion occurs a all scales of he universe, saring from he moion of elecrons in
More informationFeedback design using nonsmooth control Lyapunov functions: A numerical case study for the nonholonomic integrator
Feedback design using nonsmooh conrol Lyapunov funcions: A numerical case sudy for he nonholonomic inegraor Philipp Braun,2, Lars Grüne, Chrisopher M. Kelle 2 Absrac Theoreical resuls for he exisence of
More informationTHE 2-BODY PROBLEM. FIGURE 1. A pair of ellipses sharing a common focus. (c,b) c+a ROBERT J. VANDERBEI
THE 2-BODY PROBLEM ROBERT J. VANDERBEI ABSTRACT. In his shor noe, we show ha a pair of ellipses wih a common focus is a soluion o he 2-body problem. INTRODUCTION. Solving he 2-body problem from scrach
More informationPhysics 235 Chapter 2. Chapter 2 Newtonian Mechanics Single Particle
Chaper 2 Newonian Mechanics Single Paricle In his Chaper we will review wha Newon s laws of mechanics ell us abou he moion of a single paricle. Newon s laws are only valid in suiable reference frames,
More informationOn Boundedness of Q-Learning Iterates for Stochastic Shortest Path Problems
MATHEMATICS OF OPERATIONS RESEARCH Vol. 38, No. 2, May 2013, pp. 209 227 ISSN 0364-765X (prin) ISSN 1526-5471 (online) hp://dx.doi.org/10.1287/moor.1120.0562 2013 INFORMS On Boundedness of Q-Learning Ieraes
More informationA Hop Constrained Min-Sum Arborescence with Outage Costs
A Hop Consrained Min-Sum Arborescence wih Ouage Coss Rakesh Kawara Minnesoa Sae Universiy, Mankao, MN 56001 Email: Kawara@mnsu.edu Absrac The hop consrained min-sum arborescence wih ouage coss problem
More informationAn Introduction to Backward Stochastic Differential Equations (BSDEs) PIMS Summer School 2016 in Mathematical Finance.
1 An Inroducion o Backward Sochasic Differenial Equaions (BSDEs) PIMS Summer School 2016 in Mahemaical Finance June 25, 2016 Chrisoph Frei cfrei@ualbera.ca This inroducion is based on Touzi [14], Bouchard
More informationFor example, the comb filter generated from. ( ) has a transfer function. e ) has L notches at ω = (2k+1)π/L and L peaks at ω = 2π k/l,
Comb Filers The simple filers discussed so far are characeried eiher by a single passband and/or a single sopband There are applicaions where filers wih muliple passbands and sopbands are required The
More information2.7. Some common engineering functions. Introduction. Prerequisites. Learning Outcomes
Some common engineering funcions 2.7 Inroducion This secion provides a caalogue of some common funcions ofen used in Science and Engineering. These include polynomials, raional funcions, he modulus funcion
More informationELE 538B: Large-Scale Optimization for Data Science. Introduction. Yuxin Chen Princeton University, Spring 2018
ELE 538B: Large-Scale Opimizaion for Daa Science Inroducion Yuxin Chen Princeon Universiy, Spring 2018 Surge of daa-inensive applicaions Widespread applicaions in large-scale daa science and learning 2.5
More information