VARIATIONAL CALCULUS IN SPACE OF MEASURES AND OPTIMAL DESIGN
|
|
- Rudolf Tyler
- 6 years ago
- Views:
Transcription
1 Chapter 1 VARIATIONAL CALCULUS IN SPACE OF MEASURES AND OPTIMAL DESIGN Ilya Molchanov Department of Statistics University of Glasgow ilya@stats.gla.ac.uk ilya Sergei Zuyev Department of Statistics and Modelling Science University of Strathclyde sergei@stams.strath.ac.uk sergei Abstract Keywords: The paper applies abstract optimisation principles in the space of measures within the context of optimal design problems. It is shown that within this framework it is possible to treat various design criteria and constraints in a unified manner providing a universal variant of the Kiefer-Wolfowitz theorem and giving a full spectrum of optimality criteria for particular cases. The described steepest descent algorithm uses the true direction of the steepest descent and descends faster than the conventional sequential algorithms that involve renormalisation on every step. design of experiments, gradient methods, optimal design, regression, space of measures Introduction Consider a standard linear optimal design problem on design space (assumed to be a locally compact separable metric space) with regressors ܵ ½ ܵ ܵµ and unknown coefficients ½ µ. The optimal design is described as a probability measure on that determines frequencies 1
2 2 making an observation at particular points. The information matrix is given by Å µ ܵ ܵ ܵ (1.1) see, e. g., (Atkinson and Donev, 1992) for details. The measure that minimises Ø Å ½ µ is called D-optimal design measure. To create a convex optimisation criterion it is convenient to minimise Å µ ÐÓ Ø Å µ, which is a convex functional on the space of matrices. In general, a measure that minimises a given differentiable functional Å µµ is called a -optimal design measure. It is common to handle the above optimisation problem by first finding an optimal information matrix from the class defined by all admissible design measures and afterwards by relating it to the corresponding design measure. Schematically, such methods of finding the optimal solution involve the following two stages: Å µµ Step I Å µ Step II (1.2) Step I concerns optimisation in the Euclidean space of dimension ½µ¾, while Step II aims to identify the measure by the obtained in the previous step information matrix. However, the chain approach based on (1.2) cannot be easily amended when the formulation of the problem changes, e.g., when constraints on appear. This usually calls for major changes in proofs, since a new family of matrices has to be analysed in Step I. Therefore, for each new type of constraints on not only Step II, but also Step I, has to be reworked, for example, as in (Cook and Fedorov, 1995) and (Fedorov and Hackl, 1997), where a number of different types of constraint are analysed. Although the family of non-negative measures is not a linear space, many objective functionals can be extended onto the linear space of signed measures as the definition (1.1) of the information matrix Å µ applies literally to signed measures. This allows us to treat the design problems described above as a special case of the optimisation of functionals defined on signed measures. We show that this abstract setting incorporates naturally many specific optimal design problems and leads to new (apparently more efficient) steepest descent algorithms for numerical computation of optimal designs. Recall that the directional (or Gateaux) derivative of a real-valued functional in a Banach space is defined by ÜµÚ ÐÑ Ø ½ Ü ØÚµ ܵµ (1.3) ؼ where Ú ¾ is a direction (Hille and Phillips, 1957). The functional is said to have a Fréchet (or strong) derivative if Ü Úµ ܵ ÜµÚ Ó Úµ as Ú ¼
3 Variational calculus in space of measures and optimal design 3 where ÜµÚ is a bounded linear continuous functional of Ú. In the design context, is a functional on the space of measures, such as µ Å µµ, and is a norm defined for signed measures, e. g. the total variation norm. Then the Fréchet derivative is written as µ, where is a signed measure. Clearly, if is a probability measure, then Ø as in (1.3) above is not necessarily a probability measure or even a positive measure. Therefore, ص may not have a direct interpretation within the design framework for an arbitrary. To circumvent this difficulty, it is quite typical in the optimal design literature to replace (1.3) by the following definition of the directional derivative: µ ÐÑ Ø ½ ½ ص ص µµ (1.4) ؼ (Atkinson and Donev, 1992; Wynn, 1972). Now ½ ص Ø is a probability measure for Ø ¾ ¼ ½ if both and are probability measures. This definition of the directional derivative is used to construct the steepest (with respect to differential operator ) descent algorithms for finding optimal designs when it is not possible to obtain analytical solutions (Wynn, 1972). However, the descent related to (1.4) is only asymptotically equivalent to the true steepest descent as defined by in (1.3). Indeed, it is easy to see from the definitions that µ µ Thus, the descent direction determined by minimising µ over all with the given norm is not the true steepest descent direction obtained by minimising µ. The steepest descent algorithm described in Section 3 emerges from our theoretical results on constrained optimisation in the space of measures presented in Section 1. In contrast to the classical sequential algorithms in the optimal design literature (Wu, 1978ab; Wu and Wynn, 1978; Wynn, 1970), we do not renormalise the obtained design measure on each step. Instead, the algorithm adds a signed measure chosen so to minimise the Fréchet derivative of the goal function among all measures satisfying the imposed constraints. This extends the ideas of Atwood (1973, 1976), Fedorov (1972) and Ford (1976, pp.59 64) and puts the whole algorithm into the conventional optimisation framework. Working on the linear space of signed measures rather than on the cone of positive measures makes this algorithm just a special case of the general steepest descent algorithm known from the optimisation literature. To establish its required convergence properties it is now sufficient to refer to the general results for the steepest descent method, see (Polak, 1997). Section 1 surveys several necessary concepts of abstract optimisation for functionals defined on the cone of positive measures describing a number of
4 4 results adapted from (Molchanov and Zuyev, 1997), where proofs and further generalisations can be found. Applications to optimal designs are discussed in Section 2. Section 3 is devoted to steepest descent type algorithms. 1. OPTIMISATION IN THE SPACE OF MEASURES Let Å be the family of all non-negative finite measures on a Polish space and Å be the linear space of all signed measures with bounded total variation on its Borel sets. In numerical implementations becomes a grid in a Euclidean space Ê and ¾ Å is a non-negative array indexed by the grid s nodes. For every ¾ Å, denote by (resp. ) the positive (resp. negative) part in the Jordan decomposition of. Consider the following optimisation problem with finitely many constraints of equality and inequality type: subject to À µ ¼ À µ ¼ µ Ò (1.5) ½ Ñ Ñ ½ (1.6) It is always assumed that Å Ê and À Å Ê are Fréchet differentiable functions. Most differentiable functionals of measures met in practice have derivatives which can be represented in the integral form. We consider here only this common case, so that there exist measurable real-valued functions Ü µ and Ü µ, ½, such that for all ¾ Å µ Ü µ ܵ and À µ Ü µ ܵ (1.7) where ½ µ. Note that all integrals are over unless specified otherwise. Furthermore, assume that the solution of Problem (1.5) is regular, i.e. the functions ½ µ Ñ µ are linear independent and there exists ¾ Å such that Ê Ü µ ܵ ¼ for all ½ Ñ Ê Ü µ ܵ ¼ for all ¾ Ñ ½ verifying À µ ¼ (1.8) (without the inequality constraints, (1.8) trivially holds with being the zero measure). The regularity condition above guarantees the existence and boundedness of the Lagrange multipliers (Zowe and Kurcyusz, 1979) for the problem (1.5).
5 Variational calculus in space of measures and optimal design 5 Theorem 1 (see Molchanov and Zuyev, 1997). Let be a regular local minimum of subject to (1.6). Then there exists Ù Ù ½ Ù µ with Ù ¼ if À µ ¼ and Ù ¼ if À µ ¼ for ¾ Ñ ½, such that Ü µ Ù Ü µ almost everywhere (1.9) Ü µ Ù Ü µ for all Ü ¾ Example 1. The simplest (but extremely important) example concerns minimisation of µ over all ¾ Å with the fixed total mass µ ½. In the above framework, Ê this corresponds to the case when ½ and the only constraint is À ½ µ ܵ ½ ¼. Then (1.9) becomes Ü µ Ù almost everywhere (1.10) Ü µ Ù for all Ü ¾ In the sequel we call (1.10) a necessary condition for a minimum in the fixed total mass problem. 2. APPLICATIONS TO OPTIMAL DESIGN Optimal design problems can be naturally treated within the above described general framework of optimisation of functionals defined on finite measures. Theorem 1 directly applies to functionals of measures typical in the optimal design literature and under quite general differentiable constraints. Although we do not assume any convexity assumptions on and work exclusively with necessary optimal conditions, convexity of immediately ensures the existence of the optimal design. Theorem 1 can easily be specialised to functionals that effectively depend on the information matrix. Given a function Ê ¾ Ê, a -optimal design measure minimises µ Å µµ over all probability measures on with possibly some additional constraints. Direct computation of the derivative of leads to the following result that can be considered as a generalised Kiefer Wolfowitz theorem. Corollary 2. Let provide a -optimal design subject to constraints (1.6). Then (1.9) holds with Ü µ ܵ Å µ µ ܵ and Å µ µ Å µ Ñ µ where Ñ Ê Üµ ܵ ܵ is the µ-th entry of the information matrix.
6 6 Example 2. (D-optimal design) Put Å µ ÐÓ Ø Å. Then Å µ Å ½, the only constraint if À µ µ ½ has the derivative ½. Since Ü µ ܵŠ½ µ ܵ Ü µ, Theorem 1 yields the necessary condition in the Kiefer-Wolfowitz characterisation of D-optimal designs. Example 3. (D-optimal design with fixed moments) Let Ê and assume that along with the constraint on the total mass µ ½ we fix the expectation of, which is a vector Ñ Ê Ü Üµ. These constraints can be written as À µ Ñ ¼µ, where À µ is a ½µ-dimensional vector function with the components À µ Ê Ü Üµ for ½ and À ½ µ µ ½. Clearly, (1.7) holds with Ü µ Ü ½µ. By Theorem 1, if minimises µ under the condition À µ Ñ ¼µ, then Ü µ Ú Ü Ù Ü µ Ú Ü Ù almost everywhere for all Ü ¾ for some Ú ¾ Ê. In other words, Ü µ is affine for -almost all Ü. Example 4. (D-optimal designs with a limited cost) Let ܵ determine the cost of making an observation at point Ü. If the expected total cost is bounded by, then the design measure should, in addition to µ ½, satisfy the constraint Ê Üµ ܵ. If provides a -optimal design under such constraints, then Ü µ Ù Û Üµ Ü µ Ù Û Üµ a.e. for all Ü ¾ for some Ù ¾ Ê and Û ¼ if Ê Üµ ܵ and Û ¼ if Ê Üµ ܵ. Since the function µ Å µµ in the D-optimality problem is a convex function of, the above necessary conditions become necessary and sufficient conditions, thus providing a full characterisation of the optimal designs. Other examples, like D-optimal designs with bounded densities, design on productspaces, etc., can be treated similarly. 3. GRADIENT METHODS Direction of the steepest descent. The gradient descent method relies on knowledge of the derivative of the goal functional. As before, it is assumed that µ is Fréchet differentiable and its derivative has the integral representation (1.7). We first consider the general setup, which will be specialised later for particular constraints and objective functionals.
7 Variational calculus in space of measures and optimal design 7 The most basic method of the gradient descent type used in the optimal design suggests moving from Ò (the approximation on step Ò) to Ò ½ ½ «Ò µ Ò «Ò Ò, where ¼ «Ò ½ and Ò minimises µ over all probability measures (Wynn, 1970). It is easy to see that such Ò is concentrated at the points where the corresponding gradient function Ü µ is minimised. Rearranging the terms, we obtain Ò ½ Ò «Ò Ò Ò µ (1.11) In this form the algorithm looks like a conventional descent algorithm that descends along the direction Ò «Ò Ò Ò µ. In this particular case, the step size is Ò ¾«Ò, and Ò Ò Ò with Ò «Ò Ò Ò «Ò Ò Such a choice of Ò ensures that Ò Ò remains a probability measure since the negative part of Ò is proportional to Ò with «Ò ½. However, if we do not restrict the choice for the descent direction to measures with the negative part proportional to Ò, then it is possible to find a steeper descent direction Ò than the one suggested by (1.11). If the current value Ò, then the steepness of the descent direction is commonly characterised by the value of the derivative µ Ü µ ܵ The true steepest descent direction must be chosen to minimise µ over all signed measures with total variation ¾«Ò such that belongs to the family Å of non-negative measures and satisfies all specified constraints. For instance, if the only constraint is that the total mass µ ½, then any signed measure with µ ¼ and the negative part dominated by would ensure that stays in Å and maintains the required total mass. Consider the problem with only linear constraints of equality type: À µ ܵ ܵ ½ (1.12) where ½ are given real numbers. In vector form, À µ Ê Üµ ܵ, where À À ½ À µ, ½ µ and ½ µ implying that À µ Ê Üµ ܵ. For a ¾ Å denote by the family of all signed measures ¾ Å such that ¾ Å and satisfies the constraints (1.12). The family represents admissible directions of descent. Let denote the restriction of a measure onto a Borel set, i.e. µ µ. The following results that we give without proof characterise the steepest direction. Recall that vectors
8 8 Û ½ Û ½ are called affinely independent if Û ¾ Û ½ Û ½ Û ½ are linearly independent. Theorem 3 (see Molchanov and Zuyev, 2000). The minimum of µ over all ¾ with is achieved on a È signed measure such ½ that has at most atoms and Ø ½ for some ¼ Ø ½ with Ø ½ Ø ½ ½ and some measurable sets such that vectors À µ ½ ½, are affinely independent. Corollary 4. If the only constraint is µ ½, then the minimum of µ over all ¾ with is achieved on a signed measure such that is the positive measure of total mass ¾ concentrated on the points of the global minima of Ü µ and Šص Å µòå Ø µ, where and Å Ôµ Ü ¾ Ü µ Ô Ø ÒÔ Å Ôµµ ¾ (1.13) ÙÔÔ Å Ôµµ ¾ (1.14) The factor is chosen in so that Å Ø µµ Å µ Ò Å Ø µµ ¾. It is interesting to note that, without constraint on the total variation norm of the increment measure, the steepest direction preserving the total mass is the measure Æ Ü¼, where Æ Ü¼ is the unit measure concentrated on a global minimum point Ü ¼ of Ü µ. Thus the classical algorithm based on the modified directional derivative (1.4) uses, in fact, the scaled variant of this direction. Steepest descent in the fixed total mass problem. The necessary condition given in Corollary 2 can be used as a stopping rule for descent algorithms. Corollary 4 above that describes the steepest descent direction in the minimisation problem with a fixed total mass justifies the following algorithm. Procedure Óº ØÔ Input. Initial measure. Step 0. Compute µ. Step 1. Compute Ü µ. If ºÓÔØÑ( ), stop. Otherwise, choose the step size. Step 2. Compute ½ غ ØÔ µ. Step 3. If ½ ½ µ, then ½ ; ½ ; and go to Step 2. Otherwise, go to Step 1.
9 Variational calculus in space of measures and optimal design 9 The necessary condition for the optimum, that is used as a stopping rule in Step 1 above, is given by (1.10): the function ܵ Ü µ is constant and takes its minimal value on the support of an optimal. Strictly speaking, the support of on the discrete space is the set Ë Ü ¾ ܵ ¼. But in practice one may wish to ignore the atoms of the mass less than a certain small threshold ÙÔÔºØÓÐ. The boolean procedure ºÓÔØÑ has the following structure: Procedure ºÓÔØÑ Input. Measure, gradient function ܵ, tolerance ØÓÐ, ÙÔÔºØÓÐ. Step 1. Determine support Ë of up to tolerance ÙÔÔºØÓÐ. Step 2. If ÑÜ Ü¾Ë Üµ ÑÒ Üµ ØÓÐ return ÌÊÍ, and otherwise ÄË. The procedure غ ØÔ returns the updated measure ½, where is an increment measure with total mass 0 and variance along the steepest direction described in Corollary 4. Procedure غ ØÔ Input. Step size, measure, gradient function ܵ. Step 0. Put ½ ܵ ܵ for each point Ü ¾. Step 1. Find a point Ü ¼ of the global minimum of ܵ and put ½ Ü ¼ µ Ü ¼ µ ¾. Step 2. Find Ø and from (1.13) and (1.14), put ½ ܵ ¼ for all Ü ¾ Å Ø µ, decrease the total ½ -mass of the points from Å µ Ò Å Ø µ by value ¾ Å Ø µµ and return thus obtained ½. In Step 1 above the mass ¾ can also be spread uniformly or in any other manner over the points of the global minimum of ܵ. The described algorithm leads to a global minimum when applied to convex objective functions. In the general case it may stuck in a local minimum, the feature common for gradient algorithms applied in the context of global optimisation. There are many possible methods suitable to choose the step size in Step 1 of procedure Óº ØÔ. Many aspects can be taken into consideration: the previous step size and/or difference between the supremum and infimum of Ü µ over the support of. The Armijo method widely used for general gradient descent algorithms (Polak, 1997) defines the new step size to be Ñ, the integer Ñ is such that Ñ µ µ «Ñ ½ µ µ «Ü µ Ñ Üµ Ü µ Ñ ½ ܵ where ¼ «½ and Ñ is the steepest descent measure with the total variation Ñ described in Corollary 4.
10 10 The corresponding steepest descent algorithms are applicable for general differentiable objective functions and realised in ËÔÐÙ»Ê library Ñ Ø (for MEasures with FIxed mass STeepest Ascent/descent) from bundle Ñ ÓÔ that can be obtained from the author s web-pages. The chosen optimality criterion and the regressors should be passed as arguments to the descent procedure using (appropriately coded) functions and. A numerical example. Cubic regression through the origin. Let ¼ ½ and let Ü Ü ¾ Ü µ. For the D-optimal design in this model an exact solution is available that assigns equal weights to three points Ô µ½¼ and 1 (Atkinson and Donev, 1992, p.119). The initial measure is taken to be uniform on ¼ ½, discretised with grid size In the algorithm described in (Atkinson and Donev, 1992) and further referred to as A0, a mass equal to the step size is added into the point of minimum of the gradient function Ü µ, and the measure is then rescaled to the original total mass. The algorithm A1 is based on Corollary 4 and described above. We run both algorithms A0 and A1 until either the objective function no longer decreases or the stopping condition is satisfied: the standardised response function attains its minimum and is constant (within the predetermined tolerance level of 0.01) on the support of the current measure. It takes 28 steps for our algorithm A1 to obtain the following approximation to the optimal measure: ¼¾µ ¼¾¼, ¼¾µ ¼½¾, ¼¾µ ¼¾¾, ¼ µ ¼½¼¼, ½µ ¼. Note that the distribution of the mass over the nearest grid points corresponds exactly to the true atoms positioned at irrational points Ô µ½¼, since ¼¾µ ¼¾µ ¼ ¼¾ and ¼¾µ ¼ µ ¼ ¼¾. The final step size is , µ ½½¾½ ½¼, and the range of Ü µ on the support of is ¼¼¼¼ ¼ ØÓÐ ¼¼½. Thus, algorithm A1 achieves the optimal design measure. In contrast, it takes 87 steps for A0 to arrive at a similar result (which still has a number of support points with negligible mass, but not true zeroes). Despite the fact that our algorithm makes more calculations at each step; it took 1.96 seconds of system time on a Pentium-II PC with 256 Mb RAM running at 450 MHz under ÄÒÙÜ to finalise the task, compared to 3.70 seconds for A0. The difference becomes even more spectacular for smaller tolerance levels or finer grids. Multiple linear constraints. Although the steepest direction for optimisation with many linear constraints given by (1.12) is characterised in Theorem 3, its practical determination becomes a difficult problem. Indeed, it is easy to see that for a discrete space (used in numerical methods) minimisation of µ over all signed measures ¾ with is a linear programming problem of dimension equal to the cardinality of. Therefore, in the presence of many
11 Variational calculus in space of measures and optimal design 11 constraints, it might be computationally more efficient to use an approximation to the exact steepest direction. For instance, it is possible to fix the negative component of the increment at every step and vary its positive part. Due to Theorem 3, the positive part of the steepest increment measure consists of at most atoms. With the negative part being proportional to, we suggest moving from the current measure to where for some ¼. This is equivalent to renormalisation ¼ µ with ½ µ and ¼ ½. The measure has atoms of masses Ô ½ Ô located at points Ü ½ Ü chosen so that to minimise the directional derivative µ (or, equivalently, µ ) and to maintain the imposed constraints (1.12). The value of characterises the size of the step, although it is not equal to the total variation of. To satisfy the linear constraints À µ ½ µ we need to have À µ ½ that can be written in matrix form as Ô Ü µ À Ü ½ Ü µô with Ô Ô ½ Ô µ and À Ü ½ Ü µ Ü µ ½. This implies Ô À Ü ½ Ü µ ½ (1.15) Since minimises, the directional derivative µ is minimised if µ Ü ½ Ü µà Ü ½ Ü µ ½ (1.16) where Ü ½ Ü µ Ü ½ µ Ü µµ. The right-hand side of (1.16) is a function of variables Ü ½ Ü that should be minimised to find the optimal locations of the atoms. Their masses Ô ½ Ô are determined by (1.15). Note that minimisation is restricted to only those -tuples Ü ½ Ü that provide Ô with all non-negative components. Since this descent differs from the steepest descent given in Theorem 3, an additional analysis is necessary to ensure that the algorithm of the described kind does converge to the desired solution. By using the same arguments as in (Wu and Wynn, 1978), it is possible to prove the dichotomous theorem for this situation. The descent algorithm for many constraints based on renormalisation procedure has been programmed in the ËÔÐÙ and Ê languages. The corresponding
12 12 library Ñ (for MEasure DEscent/Ascent) from bundle Ñ ÓÔ and related examples can be obtained from the authors web-pages. In the case of a single constraint on the total mass the described approach turns into a renormalisation method: has a single atom that is placed at the point of global minimum of µ to minimise µ. References Atkinson, A. C. and Donev, A. N. (1992). Optimum Experimental Designs. Clarendon Press, Oxford. Atwood, C. L. (1973). Sequences converging to D-optimal designs of experiments. Ann. Statist., 1: Atwood, C. L. (1976). Convergent design sequences, for sufficiently regular optimality criteria. Ann. Statist., 4: Cook, D. and Fedorov, V. (1995). Constrained optimization of experimental design. Statistics, 26: Fedorov, V. V. (1972). Theory of Optimal Experiments. Academic Press, N.Y. Fedorov, V. V. and Hackl, P. (1997). Model-Oriented Design of Experiments, volume 125 of Lect. Notes Statist. Springer, New York. Ford, I. (1976). Optimal Static and Sequential Design: a Critical Review. PhD Thesis, Department of Statistics, University of Glasgow, Glasgow. Hille, E. and Phillips, R. S. (1957). Functional Analysis and Semigroups, volume XXXI of AMS Colloquium Publications. American Mathematical Society, Providence. Molchanov, I. and Zuyev, S. (1997). Variational analysis of functionals of a Poisson process. Rapport de Recherche 3302, INRIA, Sophia-Antipolis. To appear in Math. Oper. Res. Molchanov, I. and Zuyev, S. (2000). Steepest descent algorithms in space of measures. To appear. Polak, E. (1997). Optimization. Springer, New York. Wu, C.-F. (1978a). Some algorithmic aspects of the theory of optimal design. Ann. Statist., 6: Wu, C.-F. (1978b). Some iterative procedures for generating nonsingular optimal designs. Comm. Statist. Theory Methods, A7(14): Wu, C.-F. and Wynn, H. P. (1978). The convergence of general step-length algorithms for regular optimum design criteria. Ann. Statist., 6: Wynn, H. P. (1970). The sequential generation of D-optimum experimental designs. Ann. Math. Statist., 41: Wynn, H. P. (1972). Results in the theory and construction of D-optimum experimental designs. J. Roy. Statist. Soc. Ser. B, 34: Zowe, J. and Kurcyusz, S. (1979). Regularity and stability for the mathematical programming problem in Banach spaces. Appl. Math. Optim., 5:49 62.
A Generalization of Principal Component Analysis to the Exponential Family
A Generalization of Principal Component Analysis to the Exponential Family Michael Collins Sanjoy Dasgupta Robert E. Schapire AT&T Labs Research 8 Park Avenue, Florham Park, NJ 7932 mcollins, dasgupta,
More informationA Language for Task Orchestration and its Semantic Properties
DEPARTMENT OF COMPUTER SCIENCES A Language for Task Orchestration and its Semantic Properties David Kitchin, William Cook and Jayadev Misra Department of Computer Science University of Texas at Austin
More informationObservations on the Stability Properties of Cooperative Systems
1 Observations on the Stability Properties of Cooperative Systems Oliver Mason and Mark Verwoerd Abstract We extend two fundamental properties of positive linear time-invariant (LTI) systems to homogeneous
More informationMargin Maximizing Loss Functions
Margin Maximizing Loss Functions Saharon Rosset, Ji Zhu and Trevor Hastie Department of Statistics Stanford University Stanford, CA, 94305 saharon, jzhu, hastie@stat.stanford.edu Abstract Margin maximizing
More information3 Integration and Expectation
3 Integration and Expectation 3.1 Construction of the Lebesgue Integral Let (, F, µ) be a measure space (not necessarily a probability space). Our objective will be to define the Lebesgue integral R fdµ
More informationOn construction of constrained optimum designs
On construction of constrained optimum designs Institute of Control and Computation Engineering University of Zielona Góra, Poland DEMA2008, Cambridge, 15 August 2008 Numerical algorithms to construct
More informationAnalysis of Spectral Kernel Design based Semi-supervised Learning
Analysis of Spectral Kernel Design based Semi-supervised Learning Tong Zhang IBM T. J. Watson Research Center Yorktown Heights, NY 10598 Rie Kubota Ando IBM T. J. Watson Research Center Yorktown Heights,
More informationNonlinear Programming
Nonlinear Programming Kees Roos e-mail: C.Roos@ewi.tudelft.nl URL: http://www.isa.ewi.tudelft.nl/ roos LNMB Course De Uithof, Utrecht February 6 - May 8, A.D. 2006 Optimization Group 1 Outline for week
More informationNumerical optimization
Numerical optimization Lecture 4 Alexander & Michael Bronstein tosca.cs.technion.ac.il/book Numerical geometry of non-rigid shapes Stanford University, Winter 2009 2 Longest Slowest Shortest Minimal Maximal
More informationFast Fourier Transform Solvers and Preconditioners for Quadratic Spline Collocation
Fast Fourier Transform Solvers and Preconditioners for Quadratic Spline Collocation Christina C. Christara and Kit Sun Ng Department of Computer Science University of Toronto Toronto, Ontario M5S 3G4,
More informationNumerical optimization. Numerical optimization. Longest Shortest where Maximal Minimal. Fastest. Largest. Optimization problems
1 Numerical optimization Alexander & Michael Bronstein, 2006-2009 Michael Bronstein, 2010 tosca.cs.technion.ac.il/book Numerical optimization 048921 Advanced topics in vision Processing and Analysis of
More informationOptimal Routing Policy in Two Deterministic Queues
INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE Optimal Routing Policy in Two Deterministic Queues Bruno Gaujal Emmanuel Hyon N 3997 Septembre 2000 THÈME 1 ISSN 0249-6399 ISRN INRIA/RR--3997--FR+ENG
More informationInstitute for Computational Mathematics Hong Kong Baptist University
Institute for Computational Mathematics Hong Kong Baptist University ICM Research Report 08-0 How to find a good submatrix S. A. Goreinov, I. V. Oseledets, D. V. Savostyanov, E. E. Tyrtyshnikov, N. L.
More informationOptimization methods
Lecture notes 3 February 8, 016 1 Introduction Optimization methods In these notes we provide an overview of a selection of optimization methods. We focus on methods which rely on first-order information,
More informationChapter 8 Gradient Methods
Chapter 8 Gradient Methods An Introduction to Optimization Spring, 2014 Wei-Ta Chu 1 Introduction Recall that a level set of a function is the set of points satisfying for some constant. Thus, a point
More informationNarrowing confidence interval width of PAC learning risk function by algorithmic inference
Narrowing confidence interval width of PAC learning risk function by algorithmic inference Bruno Apolloni, Dario Malchiodi Dip. di Scienze dell Informazione, Università degli Studi di Milano Via Comelico
More informationHAIYUN ZHOU, RAVI P. AGARWAL, YEOL JE CHO, AND YONG SOO KIM
Georgian Mathematical Journal Volume 9 (2002), Number 3, 591 600 NONEXPANSIVE MAPPINGS AND ITERATIVE METHODS IN UNIFORMLY CONVEX BANACH SPACES HAIYUN ZHOU, RAVI P. AGARWAL, YEOL JE CHO, AND YONG SOO KIM
More informationAn EM algorithm for Batch Markovian Arrival Processes and its comparison to a simpler estimation procedure
An EM algorithm for Batch Markovian Arrival Processes and its comparison to a simpler estimation procedure Lothar Breuer (breuer@info04.uni-trier.de) University of Trier, Germany Abstract. Although the
More informationarxiv: v1 [math.dg] 19 Mar 2012
BEREZIN-TOEPLITZ QUANTIZATION AND ITS KERNEL EXPANSION XIAONAN MA AND GEORGE MARINESCU ABSTRACT. We survey recent results [33, 34, 35, 36] about the asymptotic expansion of Toeplitz operators and their
More informationUnconstrained optimization
Chapter 4 Unconstrained optimization An unconstrained optimization problem takes the form min x Rnf(x) (4.1) for a target functional (also called objective function) f : R n R. In this chapter and throughout
More informationTwo-scale homogenization of a hydrodynamic Elrod-Adams model
March 2004 1 Two-scale homogenization of a hydrodynamic Elrod-Adams model G. Bayada MAPLY CNRS UMR-5585 / LAMCOS CNRS UMR-5514, INSA Lyon, 69621 Villeurbanne cedex, France S. Martin MAPLY CNRS UMR-5585,
More informationLine Search Methods for Unconstrained Optimisation
Line Search Methods for Unconstrained Optimisation Lecture 8, Numerical Linear Algebra and Optimisation Oxford University Computing Laboratory, MT 2007 Dr Raphael Hauser (hauser@comlab.ox.ac.uk) The Generic
More informationStochastic Optimization with Inequality Constraints Using Simultaneous Perturbations and Penalty Functions
International Journal of Control Vol. 00, No. 00, January 2007, 1 10 Stochastic Optimization with Inequality Constraints Using Simultaneous Perturbations and Penalty Functions I-JENG WANG and JAMES C.
More informationCONTROLLABILITY OF NONLINEAR DISCRETE SYSTEMS
Int. J. Appl. Math. Comput. Sci., 2002, Vol.2, No.2, 73 80 CONTROLLABILITY OF NONLINEAR DISCRETE SYSTEMS JERZY KLAMKA Institute of Automatic Control, Silesian University of Technology ul. Akademicka 6,
More informationOn Semicontinuity of Convex-valued Multifunctions and Cesari s Property (Q)
On Semicontinuity of Convex-valued Multifunctions and Cesari s Property (Q) Andreas Löhne May 2, 2005 (last update: November 22, 2005) Abstract We investigate two types of semicontinuity for set-valued
More informationCONVEX OPTIMIZATION OVER POSITIVE POLYNOMIALS AND FILTER DESIGN. Y. Genin, Y. Hachez, Yu. Nesterov, P. Van Dooren
CONVEX OPTIMIZATION OVER POSITIVE POLYNOMIALS AND FILTER DESIGN Y. Genin, Y. Hachez, Yu. Nesterov, P. Van Dooren CESAME, Université catholique de Louvain Bâtiment Euler, Avenue G. Lemaître 4-6 B-1348 Louvain-la-Neuve,
More informationAn Improved Quantum Fourier Transform Algorithm and Applications
An Improved Quantum Fourier Transform Algorithm and Applications Lisa Hales Group in Logic and the Methodology of Science University of California at Berkeley hales@cs.berkeley.edu Sean Hallgren Ý Computer
More informationNumerical Optimization Professor Horst Cerjak, Horst Bischof, Thomas Pock Mat Vis-Gra SS09
Numerical Optimization 1 Working Horse in Computer Vision Variational Methods Shape Analysis Machine Learning Markov Random Fields Geometry Common denominator: optimization problems 2 Overview of Methods
More informationHITCHIN KOBAYASHI CORRESPONDENCE, QUIVERS, AND VORTICES INTRODUCTION
ËÁ Ì ÖÛÒ ËÖĐÓÒÖ ÁÒØÖÒØÓÒÐ ÁÒ ØØÙØ ÓÖ ÅØÑØÐ ÈÝ ÓÐØÞÑÒÒ ¹½¼¼ ÏÒ Ù ØÖ ÀØÒßÃÓÝ ÓÖÖ ÓÒÒ ÉÙÚÖ Ò ÎÓÖØ ÄÙ ÐÚÖÞßÓÒ ÙÐ Ç Ö ÖßÈÖ ÎÒÒ ÈÖÖÒØ ËÁ ½¾ ¾¼¼ µ ÂÒÙÖÝ ½ ¾¼¼ ËÙÓÖØ Ý Ø Ù ØÖÒ ÖÐ ÅÒ ØÖÝ Ó ÙØÓÒ ËÒ Ò ÙÐØÙÖ ÚÐÐ Ú
More informationLecture 16: Modern Classification (I) - Separating Hyperplanes
Lecture 16: Modern Classification (I) - Separating Hyperplanes Outline 1 2 Separating Hyperplane Binary SVM for Separable Case Bayes Rule for Binary Problems Consider the simplest case: two classes are
More informationETNA Kent State University
Electronic Transactions on Numerical Analysis. Volume 17, pp. 76-92, 2004. Copyright 2004,. ISSN 1068-9613. ETNA STRONG RANK REVEALING CHOLESKY FACTORIZATION M. GU AND L. MIRANIAN Ý Abstract. For any symmetric
More informationExamination paper for TMA4180 Optimization I
Department of Mathematical Sciences Examination paper for TMA4180 Optimization I Academic contact during examination: Phone: Examination date: 26th May 2016 Examination time (from to): 09:00 13:00 Permitted
More informationHomomorphism Preservation Theorem. Albert Atserias Universitat Politècnica de Catalunya Barcelona, Spain
Homomorphism Preservation Theorem Albert Atserias Universitat Politècnica de Catalunya Barcelona, Spain Structure of the talk 1. Classical preservation theorems 2. Preservation theorems in finite model
More informationSelf-Testing Polynomial Functions Efficiently and over Rational Domains
Chapter 1 Self-Testing Polynomial Functions Efficiently and over Rational Domains Ronitt Rubinfeld Madhu Sudan Ý Abstract In this paper we give the first self-testers and checkers for polynomials over
More informationIntroduction to Optimization Techniques. Nonlinear Optimization in Function Spaces
Introduction to Optimization Techniques Nonlinear Optimization in Function Spaces X : T : Gateaux and Fréchet Differentials Gateaux and Fréchet Differentials a vector space, Y : a normed space transformation
More informationNonlinear Optimization for Optimal Control
Nonlinear Optimization for Optimal Control Pieter Abbeel UC Berkeley EECS Many slides and figures adapted from Stephen Boyd [optional] Boyd and Vandenberghe, Convex Optimization, Chapters 9 11 [optional]
More informationTheorems. Theorem 1.11: Greatest-Lower-Bound Property. Theorem 1.20: The Archimedean property of. Theorem 1.21: -th Root of Real Numbers
Page 1 Theorems Wednesday, May 9, 2018 12:53 AM Theorem 1.11: Greatest-Lower-Bound Property Suppose is an ordered set with the least-upper-bound property Suppose, and is bounded below be the set of lower
More informationCONVERGENCE PROPERTIES OF COMBINED RELAXATION METHODS
CONVERGENCE PROPERTIES OF COMBINED RELAXATION METHODS Igor V. Konnov Department of Applied Mathematics, Kazan University Kazan 420008, Russia Preprint, March 2002 ISBN 951-42-6687-0 AMS classification:
More informationVariational inequalities for fixed point problems of quasi-nonexpansive operators 1. Rafał Zalas 2
University of Zielona Góra Faculty of Mathematics, Computer Science and Econometrics Summary of the Ph.D. thesis Variational inequalities for fixed point problems of quasi-nonexpansive operators 1 by Rafał
More informationOn John type ellipsoids
On John type ellipsoids B. Klartag Tel Aviv University Abstract Given an arbitrary convex symmetric body K R n, we construct a natural and non-trivial continuous map u K which associates ellipsoids to
More informationLinear & nonlinear classifiers
Linear & nonlinear classifiers Machine Learning Hamid Beigy Sharif University of Technology Fall 1396 Hamid Beigy (Sharif University of Technology) Linear & nonlinear classifiers Fall 1396 1 / 44 Table
More informationSECTION: CONTINUOUS OPTIMISATION LECTURE 4: QUASI-NEWTON METHODS
SECTION: CONTINUOUS OPTIMISATION LECTURE 4: QUASI-NEWTON METHODS HONOUR SCHOOL OF MATHEMATICS, OXFORD UNIVERSITY HILARY TERM 2005, DR RAPHAEL HAUSER 1. The Quasi-Newton Idea. In this lecture we will discuss
More informationSimulated Annealing for Constrained Global Optimization
Monte Carlo Methods for Computation and Optimization Final Presentation Simulated Annealing for Constrained Global Optimization H. Edwin Romeijn & Robert L.Smith (1994) Presented by Ariel Schwartz Objective
More informationEngineering Part IIB: Module 4F10 Statistical Pattern Processing Lecture 5: Single Layer Perceptrons & Estimating Linear Classifiers
Engineering Part IIB: Module 4F0 Statistical Pattern Processing Lecture 5: Single Layer Perceptrons & Estimating Linear Classifiers Phil Woodland: pcw@eng.cam.ac.uk Michaelmas 202 Engineering Part IIB:
More informationWorst case complexity of the optimal LLL algorithm
Worst case complexity of the optimal LLL algorithm ALI AKHAVI GREYC - Université de Caen, F-14032 Caen Cedex, France aliakhavi@infounicaenfr Abstract In this paper, we consider the open problem of the
More informationCONVERGENCE OF APPROXIMATING FIXED POINTS FOR MULTIVALUED NONSELF-MAPPINGS IN BANACH SPACES. Jong Soo Jung. 1. Introduction
Korean J. Math. 16 (2008), No. 2, pp. 215 231 CONVERGENCE OF APPROXIMATING FIXED POINTS FOR MULTIVALUED NONSELF-MAPPINGS IN BANACH SPACES Jong Soo Jung Abstract. Let E be a uniformly convex Banach space
More informationLecture Notes in Advanced Calculus 1 (80315) Raz Kupferman Institute of Mathematics The Hebrew University
Lecture Notes in Advanced Calculus 1 (80315) Raz Kupferman Institute of Mathematics The Hebrew University February 7, 2007 2 Contents 1 Metric Spaces 1 1.1 Basic definitions...........................
More informationLinear Discrimination Functions
Laurea Magistrale in Informatica Nicola Fanizzi Dipartimento di Informatica Università degli Studi di Bari November 4, 2009 Outline Linear models Gradient descent Perceptron Minimum square error approach
More informationIterative Methods for Solving A x = b
Iterative Methods for Solving A x = b A good (free) online source for iterative methods for solving A x = b is given in the description of a set of iterative solvers called templates found at netlib: http
More informationMore Powerful Tests for Homogeneity of Multivariate Normal Mean Vectors under an Order Restriction
Sankhyā : The Indian Journal of Statistics 2007, Volume 69, Part 4, pp. 700-716 c 2007, Indian Statistical Institute More Powerful Tests for Homogeneity of Multivariate Normal Mean Vectors under an Order
More informationLinear Regression. Robot Image Credit: Viktoriya Sukhanova 123RF.com
Linear Regression These slides were assembled by Eric Eaton, with grateful acknowledgement of the many others who made their course materials freely available online. Feel free to reuse or adapt these
More informationOptimisation and Operations Research
Optimisation and Operations Research Lecture 22: Linear Programming Revisited Matthew Roughan http://www.maths.adelaide.edu.au/matthew.roughan/ Lecture_notes/OORII/ School
More informationGradient Descent. Dr. Xiaowei Huang
Gradient Descent Dr. Xiaowei Huang https://cgi.csc.liv.ac.uk/~xiaowei/ Up to now, Three machine learning algorithms: decision tree learning k-nn linear regression only optimization objectives are discussed,
More informationSUBOPTIMALITY OF THE KARHUNEN-LOÈVE TRANSFORM FOR FIXED-RATE TRANSFORM CODING. Kenneth Zeger
SUBOPTIMALITY OF THE KARHUNEN-LOÈVE TRANSFORM FOR FIXED-RATE TRANSFORM CODING Kenneth Zeger University of California, San Diego, Department of ECE La Jolla, CA 92093-0407 USA ABSTRACT An open problem in
More informationThe Conjugate Gradient Method
The Conjugate Gradient Method Lecture 5, Continuous Optimisation Oxford University Computing Laboratory, HT 2006 Notes by Dr Raphael Hauser (hauser@comlab.ox.ac.uk) The notion of complexity (per iteration)
More informationPrincipal Component Analysis
Machine Learning Michaelmas 2017 James Worrell Principal Component Analysis 1 Introduction 1.1 Goals of PCA Principal components analysis (PCA) is a dimensionality reduction technique that can be used
More informationNumerical Optimization: Basic Concepts and Algorithms
May 27th 2015 Numerical Optimization: Basic Concepts and Algorithms R. Duvigneau R. Duvigneau - Numerical Optimization: Basic Concepts and Algorithms 1 Outline Some basic concepts in optimization Some
More informationThe Steepest Descent Algorithm for Unconstrained Optimization
The Steepest Descent Algorithm for Unconstrained Optimization Robert M. Freund February, 2014 c 2014 Massachusetts Institute of Technology. All rights reserved. 1 1 Steepest Descent Algorithm The problem
More information8 Numerical methods for unconstrained problems
8 Numerical methods for unconstrained problems Optimization is one of the important fields in numerical computation, beside solving differential equations and linear systems. We can see that these fields
More informationThe Skorokhod reflection problem for functions with discontinuities (contractive case)
The Skorokhod reflection problem for functions with discontinuities (contractive case) TAKIS KONSTANTOPOULOS Univ. of Texas at Austin Revised March 1999 Abstract Basic properties of the Skorokhod reflection
More informationThe Nearest Doubly Stochastic Matrix to a Real Matrix with the same First Moment
he Nearest Doubly Stochastic Matrix to a Real Matrix with the same First Moment William Glunt 1, homas L. Hayden 2 and Robert Reams 2 1 Department of Mathematics and Computer Science, Austin Peay State
More informationLeast Squares Optimization
Least Squares Optimization The following is a brief review of least squares optimization and constrained optimization techniques. I assume the reader is familiar with basic linear algebra, including the
More informationGradient Descent. Sargur Srihari
Gradient Descent Sargur srihari@cedar.buffalo.edu 1 Topics Simple Gradient Descent/Ascent Difficulties with Simple Gradient Descent Line Search Brent s Method Conjugate Gradient Descent Weight vectors
More informationTwo-Step Iteration Scheme for Nonexpansive Mappings in Banach Space
Mathematica Moravica Vol. 19-1 (2015), 95 105 Two-Step Iteration Scheme for Nonexpansive Mappings in Banach Space M.R. Yadav Abstract. In this paper, we introduce a new two-step iteration process to approximate
More informationCorrelation at Low Temperature: I. Exponential Decay
Correlation at Low Temperature: I. Exponential Decay Volker Bach FB Mathematik; Johannes Gutenberg-Universität; D-55099 Mainz; Germany; email: vbach@mathematik.uni-mainz.de Jacob Schach Møller Ý Département
More informationat time t, in dimension d. The index i varies in a countable set I. We call configuration the family, denoted generically by Φ: U (x i (t) x j (t))
Notations In this chapter we investigate infinite systems of interacting particles subject to Newtonian dynamics Each particle is characterized by its position an velocity x i t, v i t R d R d at time
More informationNumerical Optimization Prof. Shirish K. Shevade Department of Computer Science and Automation Indian Institute of Science, Bangalore
Numerical Optimization Prof. Shirish K. Shevade Department of Computer Science and Automation Indian Institute of Science, Bangalore Lecture - 13 Steepest Descent Method Hello, welcome back to this series
More informationThe Fundamental Theorem of Linear Inequalities
The Fundamental Theorem of Linear Inequalities Lecture 8, Continuous Optimisation Oxford University Computing Laboratory, HT 2006 Notes by Dr Raphael Hauser (hauser@comlab.ox.ac.uk) Constrained Optimisation
More informationMultilevel Preconditioning of Graph-Laplacians: Polynomial Approximation of the Pivot Blocks Inverses
Multilevel Preconditioning of Graph-Laplacians: Polynomial Approximation of the Pivot Blocks Inverses P. Boyanova 1, I. Georgiev 34, S. Margenov, L. Zikatanov 5 1 Uppsala University, Box 337, 751 05 Uppsala,
More informationThe Karush-Kuhn-Tucker (KKT) conditions
The Karush-Kuhn-Tucker (KKT) conditions In this section, we will give a set of sufficient (and at most times necessary) conditions for a x to be the solution of a given convex optimization problem. These
More informationSOME REMARKS ON THE SPACE OF DIFFERENCES OF SUBLINEAR FUNCTIONS
APPLICATIONES MATHEMATICAE 22,3 (1994), pp. 419 426 S. G. BARTELS and D. PALLASCHKE (Karlsruhe) SOME REMARKS ON THE SPACE OF DIFFERENCES OF SUBLINEAR FUNCTIONS Abstract. Two properties concerning the space
More informationBlock-tridiagonal matrices
Block-tridiagonal matrices. p.1/31 Block-tridiagonal matrices - where do these arise? - as a result of a particular mesh-point ordering - as a part of a factorization procedure, for example when we compute
More informationYou should be able to...
Lecture Outline Gradient Projection Algorithm Constant Step Length, Varying Step Length, Diminishing Step Length Complexity Issues Gradient Projection With Exploration Projection Solving QPs: active set
More informationTopology. Xiaolong Han. Department of Mathematics, California State University, Northridge, CA 91330, USA address:
Topology Xiaolong Han Department of Mathematics, California State University, Northridge, CA 91330, USA E-mail address: Xiaolong.Han@csun.edu Remark. You are entitled to a reward of 1 point toward a homework
More informationThe small ball property in Banach spaces (quantitative results)
The small ball property in Banach spaces (quantitative results) Ehrhard Behrends Abstract A metric space (M, d) is said to have the small ball property (sbp) if for every ε 0 > 0 there exists a sequence
More informationHarmonic Polynomials and Dirichlet-Type Problems. 1. Derivatives of x 2 n
Harmonic Polynomials and Dirichlet-Type Problems Sheldon Axler and Wade Ramey 30 May 1995 Abstract. We take a new approach to harmonic polynomials via differentiation. Surprisingly powerful results about
More informationarxiv: v1 [math.oc] 1 Jul 2016
Convergence Rate of Frank-Wolfe for Non-Convex Objectives Simon Lacoste-Julien INRIA - SIERRA team ENS, Paris June 8, 016 Abstract arxiv:1607.00345v1 [math.oc] 1 Jul 016 We give a simple proof that the
More informationNeural Networks Learning the network: Backprop , Fall 2018 Lecture 4
Neural Networks Learning the network: Backprop 11-785, Fall 2018 Lecture 4 1 Recap: The MLP can represent any function The MLP can be constructed to represent anything But how do we construct it? 2 Recap:
More informationCzechoslovak Mathematical Journal
Czechoslovak Mathematical Journal Oktay Duman; Cihan Orhan µ-statistically convergent function sequences Czechoslovak Mathematical Journal, Vol. 54 (2004), No. 2, 413 422 Persistent URL: http://dml.cz/dmlcz/127899
More informationOnly Intervals Preserve the Invertibility of Arithmetic Operations
Only Intervals Preserve the Invertibility of Arithmetic Operations Olga Kosheleva 1 and Vladik Kreinovich 2 1 Department of Electrical and Computer Engineering 2 Department of Computer Science University
More informationPROBLEMS. (b) (Polarization Identity) Show that in any inner product space
1 Professor Carl Cowen Math 54600 Fall 09 PROBLEMS 1. (Geometry in Inner Product Spaces) (a) (Parallelogram Law) Show that in any inner product space x + y 2 + x y 2 = 2( x 2 + y 2 ). (b) (Polarization
More information#A69 INTEGERS 13 (2013) OPTIMAL PRIMITIVE SETS WITH RESTRICTED PRIMES
#A69 INTEGERS 3 (203) OPTIMAL PRIMITIVE SETS WITH RESTRICTED PRIMES William D. Banks Department of Mathematics, University of Missouri, Columbia, Missouri bankswd@missouri.edu Greg Martin Department of
More informationDO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO
QUESTION BOOKLET EECS 227A Fall 2009 Midterm Tuesday, Ocotober 20, 11:10-12:30pm DO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO You have 80 minutes to complete the midterm. The midterm consists
More informationLund Institute of Technology Centre for Mathematical Sciences Mathematical Statistics
Lund Institute of Technology Centre for Mathematical Sciences Mathematical Statistics STATISTICAL METHODS FOR SAFETY ANALYSIS FMS065 ÓÑÔÙØ Ö Ü Ö Ì ÓÓØ ØÖ Ô Ð ÓÖ Ø Ñ Ò Ý Ò Ò ÐÝ In this exercise we will
More informationOn the simplest expression of the perturbed Moore Penrose metric generalized inverse
Annals of the University of Bucharest (mathematical series) 4 (LXII) (2013), 433 446 On the simplest expression of the perturbed Moore Penrose metric generalized inverse Jianbing Cao and Yifeng Xue Communicated
More informationA Reverse Technique for Lumping High Dimensional Model Representation Method
A Reverse Technique for Lumping High Dimensional Model Representation Method M. ALPER TUNGA Bahçeşehir University Department of Software Engineering Beşiktaş, 34349, İstanbul TURKEY TÜRKİYE) alper.tunga@bahcesehir.edu.tr
More informationOn nonexpansive and accretive operators in Banach spaces
Available online at www.isr-publications.com/jnsa J. Nonlinear Sci. Appl., 10 (2017), 3437 3446 Research Article Journal Homepage: www.tjnsa.com - www.isr-publications.com/jnsa On nonexpansive and accretive
More informationOptimal experimental design, an introduction, Jesús López Fidalgo
Optimal experimental design, an introduction Jesus.LopezFidalgo@uclm.es University of Castilla-La Mancha Department of Mathematics Institute of Applied Mathematics to Science and Engineering Books (just
More informationA GEOMETRIC THEORY FOR PRECONDITIONED INVERSE ITERATION APPLIED TO A SUBSPACE
A GEOMETRIC THEORY FOR PRECONDITIONED INVERSE ITERATION APPLIED TO A SUBSPACE KLAUS NEYMEYR ABSTRACT. The aim of this paper is to provide a convergence analysis for a preconditioned subspace iteration,
More informationConvex Optimization Conjugate, Subdifferential, Proximation
1 Lecture Notes, HCI, 3.11.211 Chapter 6 Convex Optimization Conjugate, Subdifferential, Proximation Bastian Goldlücke Computer Vision Group Technical University of Munich 2 Bastian Goldlücke Overview
More informationDeep Linear Networks with Arbitrary Loss: All Local Minima Are Global
homas Laurent * 1 James H. von Brecht * 2 Abstract We consider deep linear networks with arbitrary convex differentiable loss. We provide a short and elementary proof of the fact that all local minima
More informationUTILIZING PRIOR KNOWLEDGE IN ROBUST OPTIMAL EXPERIMENT DESIGN. EE & CS, The University of Newcastle, Australia EE, Technion, Israel.
UTILIZING PRIOR KNOWLEDGE IN ROBUST OPTIMAL EXPERIMENT DESIGN Graham C. Goodwin James S. Welsh Arie Feuer Milan Depich EE & CS, The University of Newcastle, Australia 38. EE, Technion, Israel. Abstract:
More informationCONTROL SYSTEMS, ROBOTICS AND AUTOMATION - Vol. VII - System Characteristics: Stability, Controllability, Observability - Jerzy Klamka
SYSTEM CHARACTERISTICS: STABILITY, CONTROLLABILITY, OBSERVABILITY Jerzy Klamka Institute of Automatic Control, Technical University, Gliwice, Poland Keywords: stability, controllability, observability,
More informationUnconstrained minimization of smooth functions
Unconstrained minimization of smooth functions We want to solve min x R N f(x), where f is convex. In this section, we will assume that f is differentiable (so its gradient exists at every point), and
More informationContents Ordered Fields... 2 Ordered sets and fields... 2 Construction of the Reals 1: Dedekind Cuts... 2 Metric Spaces... 3
Analysis Math Notes Study Guide Real Analysis Contents Ordered Fields 2 Ordered sets and fields 2 Construction of the Reals 1: Dedekind Cuts 2 Metric Spaces 3 Metric Spaces 3 Definitions 4 Separability
More informationBehaviour of Lipschitz functions on negligible sets. Non-differentiability in R. Outline
Behaviour of Lipschitz functions on negligible sets G. Alberti 1 M. Csörnyei 2 D. Preiss 3 1 Università di Pisa 2 University College London 3 University of Warwick Lars Ahlfors Centennial Celebration Helsinki,
More informationZangwill s Global Convergence Theorem
Zangwill s Global Convergence Theorem A theory of global convergence has been given by Zangwill 1. This theory involves the notion of a set-valued mapping, or point-to-set mapping. Definition 1.1 Given
More informationBindel, Fall 2016 Matrix Computations (CS 6210) Notes for
1 Iteration basics Notes for 2016-11-07 An iterative solver for Ax = b is produces a sequence of approximations x (k) x. We always stop after finitely many steps, based on some convergence criterion, e.g.
More informationPart V. 17 Introduction: What are measures and why measurable sets. Lebesgue Integration Theory
Part V 7 Introduction: What are measures and why measurable sets Lebesgue Integration Theory Definition 7. (Preliminary). A measure on a set is a function :2 [ ] such that. () = 2. If { } = is a finite
More informationStatistics 612: L p spaces, metrics on spaces of probabilites, and connections to estimation
Statistics 62: L p spaces, metrics on spaces of probabilites, and connections to estimation Moulinath Banerjee December 6, 2006 L p spaces and Hilbert spaces We first formally define L p spaces. Consider
More information