Unconstrained Multivariate Optimization
|
|
- Jeffry Chandler
- 6 years ago
- Views:
Transcription
1 Unconstrained Multivariate Optimization Multivariate optimization means optimization of a scalar function of a several variables: and has the general form: y = () min ( ) where () is a nonlinear scalar-valued function of the vector variable. Bacground Before we discuss optimization methods, we need to tal about how to characterize nonlinear, multivariable functions such as (). Consider the nd order aylor series epansion about the point 0 : If we let: ( ) ( ) ( ) ( ) ( ) + " # + # " # ( ) ( ) a = ( ) " + " b H = = " " "
2 Unconstrained Multivariate Optimization hen we can re-write the aylor series epansion as a quadratic approimation for (): and the derivatives are: ( ) a + b + H ( ) " b + H ( ) " H Hessian gradient We can describe some of the local geometric properties of () using its gradient and Hessian. If fact there are only a few possibilities for the local geometry, which can easily be differentiated by the eigenvalues of the Hessian (H). Recall that the eigenvalues of a square matri (H) are computed by finding all of the roots (λ i ) of its characteristic equation: I " H = 0
3 Unconstrained Multivariate Optimization he possible geometries are: ) if λ i < 0 ( i =,,n), the Hessian is said to be negative definite. his object has a unique maimum and is what we commonly refer to as a hill (in three dimensions). () ) if λ i > 0 ( i =,,n), the Hessian is said to be positive definite. his object has a unique minimum and is what we commonly refer to as a valley (in three dimensions). ()
4 Unconstrained Multivariate Optimization 3) if λ i < 0 ( i =,,m) and λ i > 0 ( i =m+,,n), the Hessian is said to be indefinite. his object has neither a unique maimum or minimum, and is what we commonly refer to as a saddle (in three dimensions). () 4) if λ i < 0 ( i =,,m) and λ i = 0 ( i =m+,,n), the Hessian is said to be negative semi-definite. his object does not have a unique maimum and is what we commonly refer to as a ridge (in three dimensions). ()
5 Unconstrained Multivariate Optimization 5) if λ i > 0 ( i =,,m) and λ i = 0 ( i =m+,,n), the Hessian is said to be positive semi-definite. his object does not have a unique minimum and is what we commonly refer to as a trough (in three dimensions). () A well posed problem has a unique optimum, so will will limit our discussions to either problems with positive definite Hessians (for minimization) or negative definite Hessians (for maimization). Further, we would prefer to choose units for our decision variables () so that the eigenvalues of the Hessian all have approimately the same magnitude. his will scale the problem so that the profit contours are concentric circles and will condition our optimization calculations.
6 Unconstrained Multivariate Optimization Necessary and Sufficient Conditions For a twice continuously differentiable scalar function (), a point * is an optimum if: and: * * 0 * = is positive definite (a minimum) is negative definite (a maimum) We can use these conditions directly, but it usually involves solving a set of simultaneous nonlinear equations (which is usually just as tough as the original optimization problem). Consider: ( ) A e = A hen: A = ( A) e + A( A) e = A ( A) e (+ A) and stationarity of the gradient requires that: A = A ( A) e (+ A) = 0 his is a set of very nonlinear equations in the variables.
7 Unconstrained Multivariate Optimization Eample Consider the scalar function: ( ) = or: $ ( ) = 3 + [ ] + # " & % where = [ ]. Stationarity of the gradient requires: " % = [ ] + $ = 0 # & " % ( $ " # & # $ % & " % = - = - $ # 0 & Chec the Hessian to classify the type of stationary point: " % " % = $ = 4 $ # & # 4 & he eigenvalues of the Hessian are: I " # 4 = = ( - ) = 0 $ % & # & ( % $ ( " 6 4 ) = 6, - i hus the Hessian is indefinite and the stationary point is a saddle point. his is a trivial eample, but it highlights the general procedure for direct use of the optimality conditions. )
8 Unconstrained Multivariate Optimization Lie the univariate search methods we studied earlier, multivariate optimization methods can be separated into two groups: those which depend solely on function evaluations, those which use derivative information (either analytical derivatives or numerical approimations). Regardless of which method you choose to solve a multivariate optimization problem, the general procedure will be: ) select a starting point(s), ) choose a search direction, multivariate method 3) minimize in chosen search direction, 4) repeat steps & 3 until converged. univariate search Also, successful solution of an optimization problem will require specification of a convergence criterion. Some possibilities include: + " ( ) ( ) " + # " $ % &
9 Sequential Univariate Searches erhaps the simplest multivariable search technique to implement would be a system of sequential univariate searches along some fied set of directions. Consider the two dimensional case, where the chosen search directions are parallel to the coordinate aes: * In this algorithm, you:. select a starting point 0,. select a coordinate direction (e.g. s = [0 ], or [ 0] ), 3. perform a univariate search min ( + s), 4. repeat steps and 3 alternating between search directions, until converged.
10 Sequential Univariate Searches he problem with this method is that a very large number of iterations may be required to attain a reasonable level of accuracy. If we new something about the orientation of the objective function the rate of convergence could be greatly enhanced. Consider the previous two dimensional problem, but using the independent search directions ([ ], [ -] ). * 0 Of course, finding the optimum in n steps only wors for quadratic objective functions where the Hessian is nown and each line search is eact. here are a large number of these optimization techniques which vary only in the way that the search directions are chosen.
11 Nelder-Meade Simple his approach has nothing to do with the SIMLEX method of linear programming. he method derives its name from an n-dimensional polytope (and as a result is often referred to as the polytope method). A polytope is an n-dimensional object that has n+ vertices: n polytope number of vertices line segment triangle 3 3 tetrahedron 4 M M M It is an easily implemented direct search method, that only relies on objective function evaluations. As a result, it can be robust to some types of discontinuities and so forth. However, it is often slow to converge and is not useful for larger problems (>0 variables).
12 Nelder-Meade Simple o illustrate the basic idea of the method, consider the twodimensional minimization problem: * Step : = ma (,, 3 ), define 4 as the reflection of through the centroid of the line joining and 3. 4 ma (,, 3 ), form a new polytope from the points, 3, 4. Steps through 4: repeat the procedure in Step to form new polytopes.
13 Nelder-Meade Simple We can repeat the procedure until the polytope defined by 4, 5, 6. Further iterations will cause us to flip between two polytopes. We can eliminate these problems by introducing two further operations into the method: contraction when the reflection step does not offer any improvement, epansion to accelerate convergence when the reflection step is an improvement *
14 Nelder-Meade Simple For minimization, the algorithm is: h is given by ( h ) = ma{( ),,( n+ )}, l is given by ( l ) = min{( ),,( n+ )}, calculate the centroid of the vertices other than h as: c & n - = $ +. + n %, i= i * ( ) h # " calculate reflection step: r = c - h if ( r ) < ( l ) then attempt epansion, e = c + γ ( r - c ) γ > (usually ~3, or could do a line search) if ( e ) < ( r ) the epansion was successful, form the new polytope replacing h with e. else the epansion step failed, form the new polytope replacing h with r. end if
15 Nelder-Meade Simple else if ( r ) < ma{( i ) i h } use the reflection, form the new polytope replacing h with r. else the reflection step failed, try a contraction, s = # c + ( h - c ), ( r ) " ( h ) $ % c + ( r - c ), ( r ) < ( h ) where 0 <β <, usually β =½. if ( s ) < min{( h ),( r )} use contraction, form the new polytope replacing h with s. else repeat contraction. end if. repeat the entire procedure until termination. he Nelder-Meade Simple method can be slow to converge, but it is useful for functions whose derivatives cannot be calculated or approimated (e.g. some non-smooth functions).
16 Nelder-Meade Simple Eample min r r notes epansion γ=3, failed 0 epansion γ=3, failed epansion γ=3, failed 0 0 contraction β= ½ epansion γ=3, failed
17 Gradient-Based Methods his family of optimization methods use first-order derivatives to determine a descent direction. (Recall that the gradient gives the direction of quicest increase for the objective function. hus, the negative of the gradient gives the direction of quicest decrease for the objective function.) Steepest Descent It would seem that the fastest way to find an optimum would be to always move in the direction in which the objective function decreases the fastest. Although this idea is intuitively appealing, it is very rarely true; however, with a good line search every iteration will move closer to the optimum.
18 Gradient-Based Methods he Steepest Descent algorithm is:. choose a starting point 0,. calculate the search direction: [ ] s = " 3. determine the net iterate for : = + + s by solving the univariate optimization problem: min ( + s ) 4. repeat steps & 3 until termination. Some termination criteria to consider: + ( )- ( ) + - # " $ # where ε is some small positive scalar. # # " " "
19 Gradient-Based Methods Steepest Descent Eample # min + + start at 0 = [3 3] ( = & $ % # " start at 0 = [ ] " ( ) 0 # 3 % & 4 $ 3 ( # $ 4 # % & 0 $ ( # % & $ 0 ( 0 # % & 0 $ ( # $ # % & 0 $ ( # % & $ 0 ( % & (? 8 " ( ) % & (? his eample shows that the method of Steepest Descent is very efficient for well-scaled quadratic optimization problems (the profit contours are concentric circles). his method is not nearly as efficient for non-quadratic objective functions or objective functions with very elliptical profit contours (poorly scaled). 0 0
20 Gradient-Based Methods Steepest Descent Eample # Consider the minimization problem: hen: " 0 99% min $ # 99 0 & # 0 " 99& = % $ " ( = 0 " " o develop an objective function for our line search, substitute + = + λ s into the original objective function: o simplify our wor, let H = [ ] $ 0 # 99 ( - " ) ( - & ) " ) ( + s ) = %# 99 0 ( = * $ 0 # 99 - $ 0 # 99 * $ 0 # 99, & %# ) / (. & %# ), & ( + %# 99 0 ) (, which yields: his epression is quadratic in λ, thus maing it easy to perform an eact line search: " 0 99% $ # 99 0 & 3 [ " + ] s H H H ( + ) = d = - H + H = 0 3 d - /.
21 Gradient-Based Methods Solving for the step length yields: # 0 " 99& H % " 99 0 ( = = $ 3 H 0 99 # " & % $ " 99 0 ( 3 = Starting at 0 = [ ] : # 0, 00 " 9, 998& % $ " 9, 998 0, 00 ( # 4, 000, 004 " 3, 999, 996& % $ " 3, 999, 996 4, 000, 004 ( ( ) λ
22 Gradient-Based Methods Conjugate Gradients Steepest Descent was based on the idea that the minimum will be found provided that we always move in the downhill direction. he second eample showed that the gradient does not always point in the direction of the optimum, due to the geometry of the problem. Fletcher and Reeves (964) developed a method which attempts to account for local curvature in determining the net search direction. he algorithm is:. choose a starting point 0,. set the initial search direction as: [ ] 0 s0 = " 3. determine the net iterate for : = + + s by solving the univariate optimization problem: min ( + s ) 4. calculate the net search direction as: s " " = " s " " 5. repeat steps 3 & 4 until termination. +
23 Gradient-Based Methods he manner in which the successive search directions are calculated is important. For quadratic functions, these successive search directions are conjugate with respect to the Hessian. his means that for the quadratic function: ( ) = a + b + H successive search directions satisfy: s Hs + As a result, these successive search directions have incorporated within them information about the geometry of the optimization problem. = 0 Conjugate Gradients Eample Consider again the optimization problem: " 0 99% min $ # 99 0 & We saw that Steepest Descent was slow to converge on this problem because of its poor scaling. As in the previous eample, we will use an eact line search. It can be shown that the optimal step length is: # 0 " 99& % s " 99 0 ( = - $ 0 99 # " & s % s $ " 99 0 (
24 Gradient-Based Methods Starting at 0 = [ ] : ( ) s " 0 # % & $ ( # & % $ ( # 0 % & $ 0 ( # 03 & % $ ) 97 ( #. 8809& % $ ( # 03 & % $ ) 97 ( #). 977& % $ ) ( Notice the much quicer convergence of the algorithm. For a quadratic objective function with eact line searches, the Conjugate Gradients method ehibits quadratic convergence. he quicer convergence comes at an increased computational requirement. hese include: a more comple search direction calculation, increased storage for maintaining previous search directions and gradients. hese are usually small in relation to the performance improvements.
25 Newton s Method he Conjugate Gradients method showed that there was a considerable performance gain to be realized by incorporating some problem geometry into the optimization algorithm. However, the method did this in an approimate sense. Newton s method does this by using second-order derivative information. In the multivariate case, Newton s method determines a search direction by using the Hessian to modify the gradient. he method is developed directly from the aylor Series approimation of the objective function. Recall that an objective function can be locally approimated as: hen, stationarity can be approimated as: ( ) ( ) ( ) ( ) ( ) + " # + # " # ( ) ( ) " + # = 0 Which can be used to determine an epression for calculating the net point in the minimization procedure: ( ) ( ) + = " " Notice that in this case we get both the direction and the step length for the minimization. Further, if the objective function is quadratic, the optimum is found in a single (Newton Step) without a line search.
26 Newton s Method At a given point, the algorithm for Newton s method is:. calculate the gradient at the current point,. calculate the Hessian at the current point, 3. calculate the Newton step: 4. calculate the net point: " ( ) ( ) = " # # = repeat steps through 4 until termination. Newton s Method Eample Consider once again the optimization problem: min " 0 99% $ # 99 0 & he gradient and Hessian for this eample is: # 0 " 99& # 0 " 99 & = % $ " (, = 99 0 % $ " 99 0 (
27 Newton s Method Starting at 0 = [ ] : ( ) 0 " $ % # & " % $ #( 97 & " 0 $ % 0 # 0& " 0 ( 99% $ #( 99 0 & Comparing the three methods on this problem:.5 Steepest Descent Conjugate Gradients Newton
28 Newton s Method As epected Newton s method was very effective, eventhough the problem was not well scaled. poor scaling was compensated for by using the inverse of the Hessian. Steepest Descent was very good for the first two iterations, when we were quite far from the optimum. But it slowed rapidly as: * Q " 0 he Conjugate Gradients approach avoided this difficulty by including some measure of problem geometry in the algorithm. However, the method started out using the Steepest Descent direction and built in curvature information as the algorithm progressed to the optimum. For higher-dimensional and non-quadratic objective functions, the number of iterations required for convergence can increase substantially. In general, Newton s Method will yield the best convergence properties, at the cost of increased computational load to calculate and store the Hessian. he Conjugate Gradients approach can be effective in situations where computational requirements are an issue. However, there are some other methods where the Hessian is approimated using gradient information and the approimate Hessian is used to calculate a Newton step. hese are called the Quasi-Newton methods.
29 Newton s Method As we saw in the univariate case, care must be taen when using Newton s method for comple functions: 0.5 () Since Newton s method is searching for a stationary point, it can oscillate between such points in the above situation. For such comple functions, Newton s method is often implemented with an alternative to step calculate the net point: + = + " where the step length is determined as: i) 0 < λ < (so we don t tae a full step), ii) perform a line search on λ. Generally, Newton and Newton-lie methods are preferred to other methods, the main difficulty is determining the Hessian. 0
30 Approimate Newton Methods As we saw in the univariate case the derivatives can be approimated by finite differences. In multiple dimensions this can require substantially more calculations. As an eample, consider forward difference approimation of the gradient: + *" ( + i ) # ( = ( % $ )" i & i where δ i is a perturbation in the direction of i. his approach requires additional objective function evaluation per dimension for forward or bacward differencing, and additional objective function evaluations per dimension for central differencing. o approimate the Hessian using only finite differences in the objective function could have the form: + [ ( + ) # ( )]# [ ( + ) # ( )] * " i j = ( % $ () " i" j %& i j Finite difference approimation of the Hessian requires at least an additional objective function evaluations per dimension. Some differencing schemes will give better performance than others; however, the increased computational load required for difference approimation of the second derivatives precludes the use of this approach for larger problems. )
31 Approimate Newton Methods Alternatively, the Hessian can be approimated in terms of gradient information. In the forward difference case the Hessian can be approimated as: # + ( = ) & *" i " + i % hi = i hen, we can form a finite difference approimation to the Hessian as: ~ H h he problem is that often this approimate Hessian is nonsymmetric. A symmetric approimation can be formed as: $ ~ ~ H = H + H Whichever approach is used to approimate the derivatives, the Newton step is calculated from them. In general, Newton s methods based on finite differences can produce results similar to analytical results, providing care is taen in choosing the perturbations δ i. (Recall that as the algorithm proceeds: hen small approimation errors will affect convergence and accuracy.) # [ ] = i [ ] $ # * and " 0. Some alternatives which can require fewer computations, yet build curvature information as the algorithms proceed are the Quasi-Newton family.
32 Quasi-Newton Methods his family of methods are the multivariate analogs of the univariate Secant methods. hey build second derivative (Hessian) information as the algorithm proceeds to solution, using available gradient information. Consider the aylor series epansion for the gradient of the objective function along the step direction s : ( ) = + " + K + runcating the aylor series at the second-order terms an rearranging slightly yields the so-called Quasi-Newton Condition: ( ) H + = " # " = g hus, any algorithm which satisfies this condition can build up curvature information as it proceeds from to + in the direction Δ ; however, the curvature information is gained in only one direction. hen as these algorithms proceed the from iteration-to-iteration, the approimate Hessians can differ by only a ran-one matri: H$ H$ vu Combining the Quasi-Newton Condition and the update formula yields: $ + + = + ( ) $ ( ) ( $ + + ) H = H vu = g
33 Quasi-Newton Methods Which yields a solution for the vector u of: u v and a Hessian update formula: = [ g " H$ ] [" " ] H$ = H$ $ v v g H + + " providing Δ and v are not orthogonal Unfortunately this update formula is not unique with respect to the Quasi-Newton Condition and the step vector s. We could add an additional term of wz to the update formula, where the vector w is any vector from the null space of Δ. (Recall that if w null(δ ), then Δ w = 0). hus there are a family of such Quasi-Newton methods which differ only in this additional term wz. erhaps the most successful member of the family is the Broyden-Fletcher-Goldfarb-Shanno (BFGS) update: H$ H$ " g " g + = + " g " $ H " " H$ " H$ " he BFGS update has several advantages including that it is symmetric and it can be further simplified if the step is calculated according to: $H = " $ #
34 Quasi-Newton Methods In this case the BFGS update formula simplifies to: H$ H$ " g " g # + = + + " g " " General Quasi-Newton Algorithm. Initialization: - choose a starting point 0, - choose an initial Hessian approimation (usually $H I ).. At iteration : - calculate the gradient, - calculate the search direction: - calculate the net iterate: + = + s using a line search on λ. - compute 0 = ( $ ) ( ) s = H " - update the Hessian ( ) using the method of your choice (e.g. BFGS). 3. Repeat step until termination., " g, ". $H +
35 Quasi-Newton Methods BFGS Eample Consider once again the optimization problem: " 0 99% min $ 99 0 # & Recall that the gradient for this eample is: = # 0 " 99 & % $ " 99 0 ( As before starting at 0 = [ ] : $ ( ) H s λ
36 Quasi-Newton Methods It is worth noting that the optimum is found in 3 steps (although we are very close within ). Also, notice that we have a very accurate approimation for the Hessian (to 7 significant figures) after updates. he BFGS algorithm was: not quite as efficient as Newtons method, approimately as efficient as the Conjugate Gradients approach, much more efficient than Steepest Descent. his is what should be epected for quadratic objective functions of low dimension. As the dimension of the problem increases, the Newton and Quasi-Newton methods will usually out perform the other methods. A cautionary note: he BFGS algorithm guarantees that the approimate Hessian remains positive definite (and invertible) providing that the initial matri is chosen as positive definite, in theory. However, round-off errors and sometimes the search history can cause the approimate Hessian to become badly conditioned. When this occurs, the approimate Hessian should be reset to some value (often the identity matri I).
37 Convergence During our discussions we have mentioned convergence properties of various algorithms. here are a number of ways with which convergence properties of an algorithm can be characterized. he most conventional approach is to determine the asymptotic behaviour of the sequence of distances between the iterates ( ) and the optimum ( * ), as the iterations of a particular algorithm proceed. ypically, the distance between an iterate and the optimum is defined as: - We are most interested in the behaviour of an algorithm within some neighbourhood of the optimum; hence, the asymptotic analysis. he asymptotic rate of convergence is defined as: * + lim " - - * * p = # with the asymptotic order p and asymptotic error β. In general, most algorithms show convergence properties which are within three categories: linear (0 β <, p = ), superlinear (β = 0, p = ), quadratic (β 0, p = ),
38 Convergence For the methods we have studied, it can be shown that: ) the Steepest Descent method ehibits superlinear convergence in most cases. For quadratic objective functions the asymptotic convergence properties can be shown to approach quadratic as κ(h) and linear as κ(h). ) the Conjugate Gradients method will generally show superlinear (and often nearly quadratic) convergence properties. 3) Newton s method converges quadratically. 4) Quasi-Newton methods can closely approach quadratic rates of convergence.
39 Unconstrained Multivariate Optimization As in the univariate case, there are two basic approaches: direct search (Nelder-Meade Simple,...), derivative-based (Steepest Descent, Newton s...). Choosing which method to use for a particular problem is a trade-off among: ease of implementation, convergence speed and accuracy, computational limitations. Unfortunately there is no best method in all situations. For nearly quadratic functions Newton s and Quasi-Newton methods will usually be very efficient. For nearly flat or nonsmooth objective functions, the direct search methods are often a good choice. For very large problems (more than 000 decision variables), algorithms which use only gradient information (Steepest Descent, Conjugate Gradients, Quasi- Newton, etc.) can be the only practical methods. Remember that the solution to a multivariable optimization problem requires: a function (and derivatives) that can be evaluated, an appropriate optimization method, a good starting point, a well chosen termination criterion.
40 Unconstrained Optimization roblems. For the Wastewater settling pond problem discussed in class, we found that the outlet concentration could be described as: c0( t) = c i te Assuming τ = 4 hours and c i =00 grams/liter, please do each of the following: t " a) Find the maimum outlet concentration using Matlab s fminbnd and fminunc commands. b) Determine the sensitivity of the maimum outlet concentration and time at which the maimum occurs to the assumed value of τ. c) Determine the sensitivity of the maimum outlet concentration and time at which the maimum occurs to the assumed value of c i.
41 Unconstrained Optimization roblems. he file inhibit.mat contains data from eperiments that relate the consumption rate (µ) of substrate (hr - ) in a wastewater treatment plant to the concentration of the substrate (S) in the feed (grams/litre) to the wastewater treatment plant. he relationship is epected to be: µ = K µ ma m + S + S K S Using the given data, please do each of the following: a) Estimate the model parameters µ, K m, and K using Matlab s lsqcurvefit command. b) he last data point in the supplied data is epected to be an outlier. Determine how sensitive the parameters estimates are to the suspected outlier? Constrained ultivariable NL - Until now we have only been discussing NL that do not have constraints. Most problems that we wish to solve will have some form of constraints. - i) equality constraints (material and energy balances) - ii) inequality constraints (limits on operating conditions) - hese type of problems require techniques that rely heavily on the unconstrained methods.
Chapter III. Unconstrained Univariate Optimization
1 Chapter III Unconstrained Univariate Optimization Introduction Interval Elimination Methods Polynomial Approximation Methods Newton s Method Quasi-Newton Methods 1 INTRODUCTION 2 1 Introduction Univariate
More information(One Dimension) Problem: for a function f(x), find x 0 such that f(x 0 ) = 0. f(x)
Solving Nonlinear Equations & Optimization One Dimension Problem: or a unction, ind 0 such that 0 = 0. 0 One Root: The Bisection Method This one s guaranteed to converge at least to a singularity, i not
More informationEAD 115. Numerical Solution of Engineering and Scientific Problems. David M. Rocke Department of Applied Science
EAD 115 Numerical Solution of Engineering and Scientific Problems David M. Rocke Department of Applied Science Multidimensional Unconstrained Optimization Suppose we have a function f() of more than one
More information1 Numerical optimization
Contents 1 Numerical optimization 5 1.1 Optimization of single-variable functions............ 5 1.1.1 Golden Section Search................... 6 1.1. Fibonacci Search...................... 8 1. Algorithms
More informationLecture V. Numerical Optimization
Lecture V Numerical Optimization Gianluca Violante New York University Quantitative Macroeconomics G. Violante, Numerical Optimization p. 1 /19 Isomorphism I We describe minimization problems: to maximize
More informationChapter 6. Nonlinear Equations. 6.1 The Problem of Nonlinear Root-finding. 6.2 Rate of Convergence
Chapter 6 Nonlinear Equations 6. The Problem of Nonlinear Root-finding In this module we consider the problem of using numerical techniques to find the roots of nonlinear equations, f () =. Initially we
More information1 Numerical optimization
Contents Numerical optimization 5. Optimization of single-variable functions.............................. 5.. Golden Section Search..................................... 6.. Fibonacci Search........................................
More informationGradient Descent. Dr. Xiaowei Huang
Gradient Descent Dr. Xiaowei Huang https://cgi.csc.liv.ac.uk/~xiaowei/ Up to now, Three machine learning algorithms: decision tree learning k-nn linear regression only optimization objectives are discussed,
More informationOptimization Methods
Optimization Methods Decision making Examples: determining which ingredients and in what quantities to add to a mixture being made so that it will meet specifications on its composition allocating available
More informationToday. Introduction to optimization Definition and motivation 1-dimensional methods. Multi-dimensional methods. General strategies, value-only methods
Optimization Last time Root inding: deinition, motivation Algorithms: Bisection, alse position, secant, Newton-Raphson Convergence & tradeos Eample applications o Newton s method Root inding in > 1 dimension
More informationUnconstrained optimization
Chapter 4 Unconstrained optimization An unconstrained optimization problem takes the form min x Rnf(x) (4.1) for a target functional (also called objective function) f : R n R. In this chapter and throughout
More informationMethods that avoid calculating the Hessian. Nonlinear Optimization; Steepest Descent, Quasi-Newton. Steepest Descent
Nonlinear Optimization Steepest Descent and Niclas Börlin Department of Computing Science Umeå University niclas.borlin@cs.umu.se A disadvantage with the Newton method is that the Hessian has to be derived
More informationOutline. Scientific Computing: An Introductory Survey. Optimization. Optimization Problems. Examples: Optimization Problems
Outline Scientific Computing: An Introductory Survey Chapter 6 Optimization 1 Prof. Michael. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction
More information17 Solution of Nonlinear Systems
17 Solution of Nonlinear Systems We now discuss the solution of systems of nonlinear equations. An important ingredient will be the multivariate Taylor theorem. Theorem 17.1 Let D = {x 1, x 2,..., x m
More informationAM 205: lecture 19. Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality Today: Newton s method for optimization, survey of optimization methods Optimality Conditions: Equality Constrained Case As another example of equality
More informationQuasi-Newton Methods
Newton s Method Pros and Cons Quasi-Newton Methods MA 348 Kurt Bryan Newton s method has some very nice properties: It s extremely fast, at least once it gets near the minimum, and with the simple modifications
More informationNumerical Optimization Professor Horst Cerjak, Horst Bischof, Thomas Pock Mat Vis-Gra SS09
Numerical Optimization 1 Working Horse in Computer Vision Variational Methods Shape Analysis Machine Learning Markov Random Fields Geometry Common denominator: optimization problems 2 Overview of Methods
More informationNonlinear Programming
Nonlinear Programming Kees Roos e-mail: C.Roos@ewi.tudelft.nl URL: http://www.isa.ewi.tudelft.nl/ roos LNMB Course De Uithof, Utrecht February 6 - May 8, A.D. 2006 Optimization Group 1 Outline for week
More informationStatic unconstrained optimization
Static unconstrained optimization 2 In unconstrained optimization an objective function is minimized without any additional restriction on the decision variables, i.e. min f(x) x X ad (2.) with X ad R
More informationLecture Notes: Geometric Considerations in Unconstrained Optimization
Lecture Notes: Geometric Considerations in Unconstrained Optimization James T. Allison February 15, 2006 The primary objectives of this lecture on unconstrained optimization are to: Establish connections
More informationStatistics 580 Optimization Methods
Statistics 580 Optimization Methods Introduction Let fx be a given real-valued function on R p. The general optimization problem is to find an x ɛ R p at which fx attain a maximum or a minimum. It is of
More informationConvex Optimization. Problem set 2. Due Monday April 26th
Convex Optimization Problem set 2 Due Monday April 26th 1 Gradient Decent without Line-search In this problem we will consider gradient descent with predetermined step sizes. That is, instead of determining
More informationChapter 4. Unconstrained optimization
Chapter 4. Unconstrained optimization Version: 28-10-2012 Material: (for details see) Chapter 11 in [FKS] (pp.251-276) A reference e.g. L.11.2 refers to the corresponding Lemma in the book [FKS] PDF-file
More informationMultivariate Newton Minimanization
Multivariate Newton Minimanization Optymalizacja syntezy biosurfaktantu Rhamnolipid Rhamnolipids are naturally occuring glycolipid produced commercially by the Pseudomonas aeruginosa species of bacteria.
More informationAlgorithms for Constrained Optimization
1 / 42 Algorithms for Constrained Optimization ME598/494 Lecture Max Yi Ren Department of Mechanical Engineering, Arizona State University April 19, 2015 2 / 42 Outline 1. Convergence 2. Sequential quadratic
More informationProgramming, numerics and optimization
Programming, numerics and optimization Lecture C-3: Unconstrained optimization II Łukasz Jankowski ljank@ippt.pan.pl Institute of Fundamental Technological Research Room 4.32, Phone +22.8261281 ext. 428
More informationTopic 8c Multi Variable Optimization
Course Instructor Dr. Raymond C. Rumpf Office: A 337 Phone: (915) 747 6958 E Mail: rcrumpf@utep.edu Topic 8c Multi Variable Optimization EE 4386/5301 Computational Methods in EE Outline Mathematical Preliminaries
More informationOptimization II: Unconstrained Multivariable
Optimization II: Unconstrained Multivariable CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Justin Solomon CS 205A: Mathematical Methods Optimization II: Unconstrained Multivariable 1
More informationNonlinear Optimization: What s important?
Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global
More informationECS550NFB Introduction to Numerical Methods using Matlab Day 2
ECS550NFB Introduction to Numerical Methods using Matlab Day 2 Lukas Laffers lukas.laffers@umb.sk Department of Mathematics, University of Matej Bel June 9, 2015 Today Root-finding: find x that solves
More informationBeyond Newton s method Thomas P. Minka
Beyond Newton s method Thomas P. Minka 2000 (revised 7/21/2017) Abstract Newton s method for optimization is equivalent to iteratively maimizing a local quadratic approimation to the objective function.
More informationv are uncorrelated, zero-mean, white
6.0 EXENDED KALMAN FILER 6.1 Introduction One of the underlying assumptions of the Kalman filter is that it is designed to estimate the states of a linear system based on measurements that are a linear
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted
More informationScientific Computing: An Introductory Survey
Scientific Computing: An Introductory Survey Chapter 6 Optimization Prof. Michael T. Heath Department of Computer Science University of Illinois at Urbana-Champaign Copyright c 2002. Reproduction permitted
More informationAn artificial neural networks (ANNs) model is a functional abstraction of the
CHAPER 3 3. Introduction An artificial neural networs (ANNs) model is a functional abstraction of the biological neural structures of the central nervous system. hey are composed of many simple and highly
More informationMultidisciplinary System Design Optimization (MSDO)
oday s opics Multidisciplinary System Design Optimization (MSDO) Approimation Methods Lecture 9 6 April 004 Design variable lining Reduced-Basis Methods Response Surface Approimations Kriging Variable-Fidelity
More informationImproving the Convergence of Back-Propogation Learning with Second Order Methods
the of Back-Propogation Learning with Second Order Methods Sue Becker and Yann le Cun, Sept 1988 Kasey Bray, October 2017 Table of Contents 1 with Back-Propagation 2 the of BP 3 A Computationally Feasible
More informationSTATIONARITY RESULTS FOR GENERATING SET SEARCH FOR LINEARLY CONSTRAINED OPTIMIZATION
STATIONARITY RESULTS FOR GENERATING SET SEARCH FOR LINEARLY CONSTRAINED OPTIMIZATION TAMARA G. KOLDA, ROBERT MICHAEL LEWIS, AND VIRGINIA TORCZON Abstract. We present a new generating set search (GSS) approach
More informationSECTION: CONTINUOUS OPTIMISATION LECTURE 4: QUASI-NEWTON METHODS
SECTION: CONTINUOUS OPTIMISATION LECTURE 4: QUASI-NEWTON METHODS HONOUR SCHOOL OF MATHEMATICS, OXFORD UNIVERSITY HILARY TERM 2005, DR RAPHAEL HAUSER 1. The Quasi-Newton Idea. In this lecture we will discuss
More informationMath 411 Preliminaries
Math 411 Preliminaries Provide a list of preliminary vocabulary and concepts Preliminary Basic Netwon s method, Taylor series expansion (for single and multiple variables), Eigenvalue, Eigenvector, Vector
More informationOptimization: Nonlinear Optimization without Constraints. Nonlinear Optimization without Constraints 1 / 23
Optimization: Nonlinear Optimization without Constraints Nonlinear Optimization without Constraints 1 / 23 Nonlinear optimization without constraints Unconstrained minimization min x f(x) where f(x) is
More informationOptimization II: Unconstrained Multivariable
Optimization II: Unconstrained Multivariable CS 205A: Mathematical Methods for Robotics, Vision, and Graphics Doug James (and Justin Solomon) CS 205A: Mathematical Methods Optimization II: Unconstrained
More information5 Handling Constraints
5 Handling Constraints Engineering design optimization problems are very rarely unconstrained. Moreover, the constraints that appear in these problems are typically nonlinear. This motivates our interest
More informationOPER 627: Nonlinear Optimization Lecture 14: Mid-term Review
OPER 627: Nonlinear Optimization Lecture 14: Mid-term Review Department of Statistical Sciences and Operations Research Virginia Commonwealth University Oct 16, 2013 (Lecture 14) Nonlinear Optimization
More informationNumerical Optimization: Basic Concepts and Algorithms
May 27th 2015 Numerical Optimization: Basic Concepts and Algorithms R. Duvigneau R. Duvigneau - Numerical Optimization: Basic Concepts and Algorithms 1 Outline Some basic concepts in optimization Some
More informationAM 205: lecture 19. Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods
AM 205: lecture 19 Last time: Conditions for optimality, Newton s method for optimization Today: survey of optimization methods Quasi-Newton Methods General form of quasi-newton methods: x k+1 = x k α
More informationNumerical Optimization of Partial Differential Equations
Numerical Optimization of Partial Differential Equations Part I: basic optimization concepts in R n Bartosz Protas Department of Mathematics & Statistics McMaster University, Hamilton, Ontario, Canada
More informationHigher-Order Methods
Higher-Order Methods Stephen J. Wright 1 2 Computer Sciences Department, University of Wisconsin-Madison. PCMI, July 2016 Stephen Wright (UW-Madison) Higher-Order Methods PCMI, July 2016 1 / 25 Smooth
More information5 Quasi-Newton Methods
Unconstrained Convex Optimization 26 5 Quasi-Newton Methods If the Hessian is unavailable... Notation: H = Hessian matrix. B is the approximation of H. C is the approximation of H 1. Problem: Solve min
More informationmin f(x). (2.1) Objectives consisting of a smooth convex term plus a nonconvex regularization term;
Chapter 2 Gradient Methods The gradient method forms the foundation of all of the schemes studied in this book. We will provide several complementary perspectives on this algorithm that highlight the many
More informationOptimization and Root Finding. Kurt Hornik
Optimization and Root Finding Kurt Hornik Basics Root finding and unconstrained smooth optimization are closely related: Solving ƒ () = 0 can be accomplished via minimizing ƒ () 2 Slide 2 Basics Root finding
More informationScientific Computing: Optimization
Scientific Computing: Optimization Aleksandar Donev Courant Institute, NYU 1 donev@courant.nyu.edu 1 Course MATH-GA.2043 or CSCI-GA.2112, Spring 2012 March 8th, 2011 A. Donev (Courant Institute) Lecture
More informationNonlinearOptimization
1/35 NonlinearOptimization Pavel Kordík Department of Computer Systems Faculty of Information Technology Czech Technical University in Prague Jiří Kašpar, Pavel Tvrdík, 2011 Unconstrained nonlinear optimization,
More informationNumerical Methods. Root Finding
Numerical Methods Solving Non Linear 1-Dimensional Equations Root Finding Given a real valued function f of one variable (say ), the idea is to find an such that: f() 0 1 Root Finding Eamples Find real
More informationA Primer on Multidimensional Optimization
A Primer on Multidimensional Optimization Prof. Dr. Florian Rupp German University of Technology in Oman (GUtech) Introduction to Numerical Methods for ENG & CS (Mathematics IV) Spring Term 2016 Eercise
More informationChapter 8 Gradient Methods
Chapter 8 Gradient Methods An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Introduction Recall that a level set of a function is the set of points satisfying for some constant. Thus, a point
More informationSearch Directions for Unconstrained Optimization
8 CHAPTER 8 Search Directions for Unconstrained Optimization In this chapter we study the choice of search directions used in our basic updating scheme x +1 = x + t d. for solving P min f(x). x R n All
More informationGeometry optimization
Geometry optimization Trygve Helgaker Centre for Theoretical and Computational Chemistry Department of Chemistry, University of Oslo, Norway European Summer School in Quantum Chemistry (ESQC) 211 Torre
More informationReview of Optimization Basics
Review of Optimization Basics. Introduction Electricity markets throughout the US are said to have a two-settlement structure. The reason for this is that the structure includes two different markets:
More information2. Quasi-Newton methods
L. Vandenberghe EE236C (Spring 2016) 2. Quasi-Newton methods variable metric methods quasi-newton methods BFGS update limited-memory quasi-newton methods 2-1 Newton method for unconstrained minimization
More informationNumerical Optimization
Numerical Optimization Unit 2: Multivariable optimization problems Che-Rung Lee Scribe: February 28, 2011 (UNIT 2) Numerical Optimization February 28, 2011 1 / 17 Partial derivative of a two variable function
More information8 Numerical methods for unconstrained problems
8 Numerical methods for unconstrained problems Optimization is one of the important fields in numerical computation, beside solving differential equations and linear systems. We can see that these fields
More informationPerformance Surfaces and Optimum Points
CSC 302 1.5 Neural Networks Performance Surfaces and Optimum Points 1 Entrance Performance learning is another important class of learning law. Network parameters are adjusted to optimize the performance
More informationNumerical Methods in Informatics
Numerical Methods in Informatics Lecture 2, 30.09.2016: Nonlinear Equations in One Variable http://www.math.uzh.ch/binf4232 Tulin Kaman Institute of Mathematics, University of Zurich E-mail: tulin.kaman@math.uzh.ch
More information, b = 0. (2) 1 2 The eigenvectors of A corresponding to the eigenvalues λ 1 = 1, λ 2 = 3 are
Quadratic forms We consider the quadratic function f : R 2 R defined by f(x) = 2 xt Ax b T x with x = (x, x 2 ) T, () where A R 2 2 is symmetric and b R 2. We will see that, depending on the eigenvalues
More informationOptimization Tutorial 1. Basic Gradient Descent
E0 270 Machine Learning Jan 16, 2015 Optimization Tutorial 1 Basic Gradient Descent Lecture by Harikrishna Narasimhan Note: This tutorial shall assume background in elementary calculus and linear algebra.
More informationData Mining (Mineria de Dades)
Data Mining (Mineria de Dades) Lluís A. Belanche belanche@lsi.upc.edu Soft Computing Research Group Dept. de Llenguatges i Sistemes Informàtics (Software department) Universitat Politècnica de Catalunya
More information0.1. Linear transformations
Suggestions for midterm review #3 The repetitoria are usually not complete; I am merely bringing up the points that many people didn t now on the recitations Linear transformations The following mostly
More informationExploring the energy landscape
Exploring the energy landscape ChE210D Today's lecture: what are general features of the potential energy surface and how can we locate and characterize minima on it Derivatives of the potential energy
More informationMORE CURVE SKETCHING
Mathematics Revision Guides More Curve Sketching Page of 3 MK HOME TUITION Mathematics Revision Guides Level: AS / A Level MEI OCR MEI: C4 MORE CURVE SKETCHING Version : 5 Date: 05--007 Mathematics Revision
More informationg(x,y) = c. For instance (see Figure 1 on the right), consider the optimization problem maximize subject to
1 of 11 11/29/2010 10:39 AM From Wikipedia, the free encyclopedia In mathematical optimization, the method of Lagrange multipliers (named after Joseph Louis Lagrange) provides a strategy for finding the
More informationMethods for Unconstrained Optimization Numerical Optimization Lectures 1-2
Methods for Unconstrained Optimization Numerical Optimization Lectures 1-2 Coralia Cartis, University of Oxford INFOMM CDT: Modelling, Analysis and Computation of Continuous Real-World Problems Methods
More informationIterative Methods for Solving A x = b
Iterative Methods for Solving A x = b A good (free) online source for iterative methods for solving A x = b is given in the description of a set of iterative solvers called templates found at netlib: http
More informationReview of Classical Optimization
Part II Review of Classical Optimization Multidisciplinary Design Optimization of Aircrafts 51 2 Deterministic Methods 2.1 One-Dimensional Unconstrained Minimization 2.1.1 Motivation Most practical optimization
More informationIntroduction to Geometry Optimization. Computational Chemistry lab 2009
Introduction to Geometry Optimization Computational Chemistry lab 9 Determination of the molecule configuration H H Diatomic molecule determine the interatomic distance H O H Triatomic molecule determine
More informationLecture 7: Minimization or maximization of functions (Recipes Chapter 10)
Lecture 7: Minimization or maximization of functions (Recipes Chapter 10) Actively studied subject for several reasons: Commonly encountered problem: e.g. Hamilton s and Lagrange s principles, economics
More informationLecture 16: October 22
0-725/36-725: Conve Optimization Fall 208 Lecturer: Ryan Tibshirani Lecture 6: October 22 Scribes: Nic Dalmasso, Alan Mishler, Benja LeRoy Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationNumerical solutions of nonlinear systems of equations
Numerical solutions of nonlinear systems of equations Tsung-Ming Huang Department of Mathematics National Taiwan Normal University, Taiwan E-mail: min@math.ntnu.edu.tw August 28, 2011 Outline 1 Fixed points
More informationIntroduction to unconstrained optimization - direct search methods
Introduction to unconstrained optimization - direct search methods Jussi Hakanen Post-doctoral researcher jussi.hakanen@jyu.fi Structure of optimization methods Typically Constraint handling converts the
More informationUNCONSTRAINED OPTIMIZATION
UNCONSTRAINED OPTIMIZATION 6. MATHEMATICAL BASIS Given a function f : R n R, and x R n such that f(x ) < f(x) for all x R n then x is called a minimizer of f and f(x ) is the minimum(value) of f. We wish
More informationCLASS NOTES Computational Methods for Engineering Applications I Spring 2015
CLASS NOTES Computational Methods for Engineering Applications I Spring 2015 Petros Koumoutsakos Gerardo Tauriello (Last update: July 2, 2015) IMPORTANT DISCLAIMERS 1. REFERENCES: Much of the material
More informationOptimization Methods: Optimization using Calculus - Equality constraints 1. Module 2 Lecture Notes 4
Optimization Methods: Optimization using Calculus - Equality constraints Module Lecture Notes 4 Optimization of Functions of Multiple Variables subect to Equality Constraints Introduction In the previous
More informationComparative study of Optimization methods for Unconstrained Multivariable Nonlinear Programming Problems
International Journal of Scientific and Research Publications, Volume 3, Issue 10, October 013 1 ISSN 50-3153 Comparative study of Optimization methods for Unconstrained Multivariable Nonlinear Programming
More informationGradient-Based Optimization
Multidisciplinary Design Optimization 48 Chapter 3 Gradient-Based Optimization 3. Introduction In Chapter we described methods to minimize (or at least decrease) a function of one variable. While problems
More informationIntroduction to gradient descent
6-1: Introduction to gradient descent Prof. J.C. Kao, UCLA Introduction to gradient descent Derivation and intuitions Hessian 6-2: Introduction to gradient descent Prof. J.C. Kao, UCLA Introduction Our
More informationPhysics 403. Segev BenZvi. Numerical Methods, Maximum Likelihood, and Least Squares. Department of Physics and Astronomy University of Rochester
Physics 403 Numerical Methods, Maximum Likelihood, and Least Squares Segev BenZvi Department of Physics and Astronomy University of Rochester Table of Contents 1 Review of Last Class Quadratic Approximation
More informationMATHEMATICS FOR COMPUTER VISION WEEK 8 OPTIMISATION PART 2. Dr Fabio Cuzzolin MSc in Computer Vision Oxford Brookes University Year
MATHEMATICS FOR COMPUTER VISION WEEK 8 OPTIMISATION PART 2 1 Dr Fabio Cuzzolin MSc in Computer Vision Oxford Brookes University Year 2013-14 OUTLINE OF WEEK 8 topics: quadratic optimisation, least squares,
More informationUNCONSTRAINED OPTIMIZATION PAUL SCHRIMPF OCTOBER 24, 2013
PAUL SCHRIMPF OCTOBER 24, 213 UNIVERSITY OF BRITISH COLUMBIA ECONOMICS 26 Today s lecture is about unconstrained optimization. If you re following along in the syllabus, you ll notice that we ve skipped
More informationLecture 18: November Review on Primal-dual interior-poit methods
10-725/36-725: Convex Optimization Fall 2016 Lecturer: Lecturer: Javier Pena Lecture 18: November 2 Scribes: Scribes: Yizhu Lin, Pan Liu Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationShiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 3. Gradient Method
Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 3 Gradient Method Shiqian Ma, MAT-258A: Numerical Optimization 2 3.1. Gradient method Classical gradient method: to minimize a differentiable convex
More informationVasil Khalidov & Miles Hansard. C.M. Bishop s PRML: Chapter 5; Neural Networks
C.M. Bishop s PRML: Chapter 5; Neural Networks Introduction The aim is, as before, to find useful decompositions of the target variable; t(x) = y(x, w) + ɛ(x) (3.7) t(x n ) and x n are the observations,
More informationConvex Optimization CMU-10725
Convex Optimization CMU-10725 Quasi Newton Methods Barnabás Póczos & Ryan Tibshirani Quasi Newton Methods 2 Outline Modified Newton Method Rank one correction of the inverse Rank two correction of the
More informationLinear Discriminant Functions
Linear Discriminant Functions Linear discriminant functions and decision surfaces Definition It is a function that is a linear combination of the components of g() = t + 0 () here is the eight vector and
More informationStatistical Geometry Processing Winter Semester 2011/2012
Statistical Geometry Processing Winter Semester 2011/2012 Linear Algebra, Function Spaces & Inverse Problems Vector and Function Spaces 3 Vectors vectors are arrows in space classically: 2 or 3 dim. Euclidian
More informationOrder of convergence. MA3232 Numerical Analysis Week 3 Jack Carl Kiefer ( ) Question: How fast does x n
Week 3 Jack Carl Kiefer (94-98) Jack Kiefer was an American statistician. Much of his research was on the optimal design of eperiments. However, he also made significant contributions to other areas of
More informationDeep Learning. Authors: I. Goodfellow, Y. Bengio, A. Courville. Chapter 4: Numerical Computation. Lecture slides edited by C. Yim. C.
Chapter 4: Numerical Computation Deep Learning Authors: I. Goodfellow, Y. Bengio, A. Courville Lecture slides edited by 1 Chapter 4: Numerical Computation 4.1 Overflow and Underflow 4.2 Poor Conditioning
More informationComputer Problems for Taylor Series and Series Convergence
Computer Problems for Taylor Series and Series Convergence The two problems below are a set; the first should be done without a computer and the second is a computer-based follow up. 1. The drawing below
More informationAM 205: lecture 18. Last time: optimization methods Today: conditions for optimality
AM 205: lecture 18 Last time: optimization methods Today: conditions for optimality Existence of Global Minimum For example: f (x, y) = x 2 + y 2 is coercive on R 2 (global min. at (0, 0)) f (x) = x 3
More informationOptimization Methods
Optimization Methods Categorization of Optimization Problems Continuous Optimization Discrete Optimization Combinatorial Optimization Variational Optimization Common Optimization Concepts in Computer Vision
More informationHandling Nonpositive Curvature in a Limited Memory Steepest Descent Method
Handling Nonpositive Curvature in a Limited Memory Steepest Descent Method Fran E. Curtis and Wei Guo Department of Industrial and Systems Engineering, Lehigh University, USA COR@L Technical Report 14T-011-R1
More information