Techinical Proofs for Nonlinear Learning using Local Coordinate Coding
|
|
- Bertina Hamilton
- 6 years ago
- Views:
Transcription
1 Techinical Proofs for Nonlinear Learning using Local Coordinate Coding 1 Notations and Main Results Denition 1.1 (Lipschitz Smoothness) A function f(x) on R d is (α, β, p)-lipschitz smooth with respect to a norm if f(x ) f(x) α x x, and where we assume α, β > 0 and p (0, 1]. f(x ) f(x) f(x) (x x) β x x 1+p, Denition 1.2 (Coordinate Coding) A coordinate coding is a pair (γ, C), where C R d is a set of anchor points, and γ is a map of x R d to γ v (x)] R C such that v γ v(x) = 1. It induces the following physical approximation of x in R d : γ(x) = γ v (x)v. Moreover, for all x R d, we dene the coding norm as ( ) 1/2 x γ = γ v (x) 2. Proposition 1.1 The map x γ v(x)v is invariant under any shift of the origin for representing data points in R d if and only if v γ v(x) = 1. Lemma 1.1 (Linearization) Let (γ, C) be an arbitrary coordinate coding on R d. (α, β, p)-lipschitz smooth function. We have for all x R d : f(x) γ v (x)f(v) α x γ(x) + β γ v (x) v γ(x) 1+p. Let f be an Denition 1.3 (Localization Measure) Given α, β, p, and coding (γ, C), we dene Q α,β,p (γ, C) = E x α x γ(x) + β ] γ v (x) v γ(x) 1+p. 1
2 Denition 1.4 (Manifold) A subset M R d is called a p-smooth (p > 0) manifold with intrinsic dimensionality m = m(m) if there exists a constant c p (M) such that given any x M, there exists m vectors v 1 (x),..., v m (x) R d so that x M: m inf γ R m x x γ j v j (x) c p(m) x x 1+p. Denition 1.5 (Covering Number) Given any subset M R d, and ɛ > 0. The covering number, denoted as N (ɛ, M), is the smallest cardinality of an ɛ-cover C M. That is, sup x M inf x v ɛ. Theorem 1.1 (Manifold Coding) If the data points x lie on a compact p-smooth manifold M, and the norm is dened as x = (x Ax) 1/2 for some positive denite matrix A. Then given any ɛ > 0, there exist anchor points C M and coding γ such that C (1 + m(m))n (ɛ, M), Q α,β,p (γ, C) αc p (M) + (1 + m p m)β] ɛ 1+p. Moreover, for all x M, we have x 2 γ 1 + (1 + m) 2. Given a local-coordinate coding scheme (γ, C), we approximate each f(x) F a α,β,p by f(x) f γ,c (ŵ, x) = ŵ v γ v (x), where we estimate the coecients using ridge regression as: n ŵ v ] = arg min φ (f γ,c (w, x i ), y i ) + λ ] (w v g(v)) 2, (1) w v] i=1 Theorem 1.2 (Generalization Bound) Suppose φ(p, y) is Lipschitz: φ 1 (p, y) B. Consider coordinate coding (γ, C), and the estimation method (1) with random training examples S n = {(x 1, y 1 ),..., (x n, y n )}. Then the expected generalization error satises the inequality: E Sn E x,y φ(f γ,c (ŵ, x), y) inf f F α,β,p E x,y φ (f(x), y) + λ (f(v) g(v)) 2 ] + B2 2λn E x x 2 γ + BQ α,β,p (γ, C). Theorem 1.3 (Consistency) Suppose the data lie on a compact manifold M R d, and the norm is the Euclidean norm in R d. If loss function φ(p, y) is Lipschitz. As n, we choose α, β, α/n, β/n 0 (α, β depends on n), and p = 0. Then it is possible to nd coding (γ, C) using unlabeled data such that C /n 0 and Q α,β,p (γ, C) 0. If we pick λn, and λ C 0. Then the local coordinate coding method (1) with g(v) 0 is consistent as n : lim E S n n E x,y φ(f(ŵ, x), y) = inf E x,yφ (f(x), y). f:m R 2
3 2 Proofs 2.1 Proof of Proposition 1.1 Consider a change of the R d origin by u R d, which shifts any point x R d to x + u, and points v C to v + u. The shift-invariance requirement implies that after the change, we map x + u to γ v(x)v + u, which should equal γ v(x)(v + u). This is equivalent to u = γ v(x)u, which holds if and only if γ v(x) = Proof of Lemma 1.1 For simplicity, let γ v = γ v (x) and x = γ(x) = γ vv. We have f(x) γ v f(v) f(x) f(x ) + γ v (f(v) f(x )) = f(x) f(x ) + γ v (f(v) f(x ) f(x ) (v x )) This implies the bound. 2.3 Proof of Theorem 1.1 f(x) f(x ) + α x x 2 + β γ v (f(v) f(x ) f(x ) (v x )) γ v x v 1+p. Let m = m(m). Given any ɛ > 0, consider an ɛ-cover C of M with C N (ɛ, M). Given each u C, dene C u = {v 1 (u),..., v d (u)}, where v j (u) are dened in Denition 1.4. Dene the anchor points as C = u C {u + v j (u) : j = 1,..., m} C. It follows that C (1 + m)n (ɛ, M). In the following, we only need to prove the existence of a coding γ on M that satises the requirement of the theorem. Without loss of generality, we assume that v j (u) = ɛ for each u and j, and given u, {v j (u) : j = 1,..., m} are orthogonal with respect to A: vj (u)av k(u) = 0 when j k. For each x M, let u x C be the closest point to x in C. We have x u x ɛ by the denition of C. Now, Denition 1.4 implies that there exists γ j (x) (j = 1,..., m) such that m x u x γ j(x)v j (u x ) c p(m)ɛ 1+p. The optimal choice is the A-projection of x u x to the subspace spanned by {v j (u x ) : j = 1,..., m}. The orthogonality condition thus implies that m γ j(x) 2 v j (u x ) 2 x u x 2 ɛ 2. 3
4 Therefore which implies that for all x: m γ j(x) 2 1, m γ j(x) m. We can now dene the coordinate coding of x M as γ j v = u x + v j (u x ) γ v (x) = 1 m γ j v = u x 0 otherwise. This implies the following bounds: and x γ(x) c p (M)ɛ 1+p 1 j = 1 m γ j v γ(x) 1+p = γ ux (x) γ(x) u x + where we have used v u x = ɛ, and γ(x) u x x u x ɛ. 2.4 Proof of Theorem 1.2 m γ j(x) (v u x ) (γ(x) u x ) 1+p (1 + m m)ɛ 1+p γ j(x) (ɛ + ɛ) 1+p (3) =1 + m p m]ɛ 1+p. (4) Consider n + 1 samples S = {(x 1, y 1 ),..., (x, y )}. We shall introduce the following notation: 1 w v ] = arg min φ (f γ,c (w, x i ), y i ) + λ ] wv 2. (5) w v] n i=1 Let k be an integer randomly drawn from {1,..., n + 1}. Let ŵ v (k) ] be the solution of ŵ v (k) ] = arg min 1 φ (f γ,c (w, x i ), y i ) + λ w 2 v, w v] n i=1,...,;i k with the k-th example left-out. We have the following stability lemma from 1], which can be stated as follows using our terminology: (2) 4
5 Lemma 2.1 The following inequality holds f γ,c (ŵ (k), x k ) f γ,c ( w, x k ) x k 2 γ 2λn φ 1(f γ,c ( w, x k ), y k ). By using Lemma 2.1, we obtain for all α > 0: φ(f γ,c ( w, x k ), y k ) φ(f γ,c (ŵ (k), x k ), y k ) =φ(f γ,c ( w, x k ), y k ) φ(f γ,c (ŵ (k), x k ), y k ) φ 1(f γ,c (ŵ (k), x k ), y k )(f γ,c ( w, x k ) f γ,c (ŵ (k), x k )) + φ 1(f γ,c (ŵ (k), x k ), y k )(f γ,c ( w, x k ) f γ,c (ŵ (k), x k )) φ 1(f γ,c (ŵ (k), x k ), y k )(f γ,c ( w, x k ) f γ,c (ŵ (k), x k )) φ 1(f γ,c (ŵ (k), x k ), y k ) 2 x k 2 γ/(2λn) B 2 x k 2 γ/(2λn). In the above derivation, the rst inequality uses the convexity of φ(f, y) with respect to f, which implies that φ(f 1, y) φ(f 2, y) φ 1 (f 2, y)(f 1 f 2 ) 0. The second inequality uses Lemma 2.1, and the third inequality uses the assumption of the loss function. Now by summing over k, and consider any xed f F α,β,p, we obtain: φ(f γ,c (ŵ (k), x k ), y k ) n n φ(f γ,c ( w, x k ), y k ) + B ] 2λn x k 2 γ ( ) 1 φ γ v (x k )f(v), y k + λ ] f(v) 2 n 1 (f(x k ), y k ) + BQ(x k )] + λ n φ ] f(v) 2 + B2 2λn + B2 2λn x k 2 γ x k 2 γ, where Q(x) = α x γ(x) + β γ v(x) v γ(x) 1+p. In the above derivation, the second inequality follows from the denition of w as the minimizer of (5). The third inequality follows from Lemma 1.1. Now by taking expectation with respect to S, we obtain (n + 1)E S φ(f γ,c (ŵ (), x ), y ) n + 1 n n E x,yφ (f(x), y) + n + 1 n BQ α,β,p(γ, C) + λ ] f(v) 2 This implies the desired bound. 2.5 Proof of Theorem B2 (n + 1) E x x 2 2λn γ. Note that any measurable function f : M R can be approximated by F α,β,p with α, β and p = 0. Therefore we only need to show lim E S n n E x,y φ(f γ,c (ŵ, x), y) = lim inf E x,y φ (f(x), y). n f F α,β,p 5
6 Theorem 1.1 implies that it is possible to pick (γ, C) such that C /n 0 and Q α,β,p (γ, C) 0. Moreover, x γ is bounded. Given any f F α,β,0 and any n independent xed A > 0; if we let f A (x) = max(min(f(x), A), A), then it is clear that f A (x) F α,α+β,0. Therefore Theorem 1.2 implies that as n, E Sn E x,y φ(f γ,c (ŵ, x), y) E x,y φ (f A (x), y) + o(1). Since A is arbitrary, we let A to obtain the desired result. References 1] Tong Zhang. Leave-one-out bounds for kernel methods. Neural Computation, 15: ,
Nonlinear Learning using Local Coordinate Coding
Nonlinear Learning using Local Coordinate Coding Kai Yu NEC Laboratories America kyu@sv.nec-labs.com Tong Zhang Rutgers University tzhang@stat.rutgers.edu Yihong Gong NEC Laboratories America ygong@sv.nec-labs.com
More informationLinear Regression and Its Applications
Linear Regression and Its Applications Predrag Radivojac October 13, 2014 Given a data set D = {(x i, y i )} n the objective is to learn the relationship between features and the target. We usually start
More informationImproved Local Coordinate Coding using Local Tangents
Improved Local Coordinate Coding using Local Tangents Kai Yu NEC Laboratories America, 10081 N. Wolfe Road, Cupertino, CA 95129 Tong Zhang Rutgers University, 110 Frelinghuysen Road, Piscataway, NJ 08854
More informationRichard DiSalvo. Dr. Elmer. Mathematical Foundations of Economics. Fall/Spring,
The Finite Dimensional Normed Linear Space Theorem Richard DiSalvo Dr. Elmer Mathematical Foundations of Economics Fall/Spring, 20-202 The claim that follows, which I have called the nite-dimensional normed
More informationSTATISTICAL BEHAVIOR AND CONSISTENCY OF CLASSIFICATION METHODS BASED ON CONVEX RISK MINIMIZATION
STATISTICAL BEHAVIOR AND CONSISTENCY OF CLASSIFICATION METHODS BASED ON CONVEX RISK MINIMIZATION Tong Zhang The Annals of Statistics, 2004 Outline Motivation Approximation error under convex risk minimization
More information1 Solutions to selected problems
1 Solutions to selected problems 1. Let A B R n. Show that int A int B but in general bd A bd B. Solution. Let x int A. Then there is ɛ > 0 such that B ɛ (x) A B. This shows x int B. If A = [0, 1] and
More information' '-'in.-i 1 'iritt in \ rrivfi pr' 1 p. ru
V X X Y Y 7 VY Y Y F # < F V 6 7»< V q q $ $» q & V 7» Q F Y Q 6 Q Y F & Q &» & V V» Y V Y [ & Y V» & VV & F > V } & F Q \ Q \» Y / 7 F F V 7 7 x» > QX < #» > X >» < F & V F» > > # < q V 6 & Y Y q < &
More informationProblem Set 6: Solutions Math 201A: Fall a n x n,
Problem Set 6: Solutions Math 201A: Fall 2016 Problem 1. Is (x n ) n=0 a Schauder basis of C([0, 1])? No. If f(x) = a n x n, n=0 where the series converges uniformly on [0, 1], then f has a power series
More informationl(y j ) = 0 for all y j (1)
Problem 1. The closed linear span of a subset {y j } of a normed vector space is defined as the intersection of all closed subspaces containing all y j and thus the smallest such subspace. 1 Show that
More informationMathematics 530. Practice Problems. n + 1 }
Department of Mathematical Sciences University of Delaware Prof. T. Angell October 19, 2015 Mathematics 530 Practice Problems 1. Recall that an indifference relation on a partially ordered set is defined
More informationMidterm 1. Every element of the set of functions is continuous
Econ 200 Mathematics for Economists Midterm Question.- Consider the set of functions F C(0, ) dened by { } F = f C(0, ) f(x) = ax b, a A R and b B R That is, F is a subset of the set of continuous functions
More informationCS229T/STATS231: Statistical Learning Theory. Lecturer: Tengyu Ma Lecture 11 Scribe: Jongho Kim, Jamie Kang October 29th, 2018
CS229T/STATS231: Statistical Learning Theory Lecturer: Tengyu Ma Lecture 11 Scribe: Jongho Kim, Jamie Kang October 29th, 2018 1 Overview This lecture mainly covers Recall the statistical theory of GANs
More informationEC 521 MATHEMATICAL METHODS FOR ECONOMICS. Lecture 1: Preliminaries
EC 521 MATHEMATICAL METHODS FOR ECONOMICS Lecture 1: Preliminaries Murat YILMAZ Boğaziçi University In this lecture we provide some basic facts from both Linear Algebra and Real Analysis, which are going
More informationApproximation by ridge functions with weights in a specified set
Approximation by ridge functions with weights in a specified set David Stewart & Palle Jorgensen University of Iowa Mathematics September 11, 2018 Extreme Learning Machines Extreme Learning Machines are
More informationSecond Order Elliptic PDE
Second Order Elliptic PDE T. Muthukumar tmk@iitk.ac.in December 16, 2014 Contents 1 A Quick Introduction to PDE 1 2 Classification of Second Order PDE 3 3 Linear Second Order Elliptic Operators 4 4 Periodic
More informationLinear Analysis Lecture 5
Linear Analysis Lecture 5 Inner Products and V Let dim V < with inner product,. Choose a basis B and let v, w V have coordinates in F n given by x 1. x n and y 1. y n, respectively. Let A F n n be the
More informationLinear Algebra Massoud Malek
CSUEB Linear Algebra Massoud Malek Inner Product and Normed Space In all that follows, the n n identity matrix is denoted by I n, the n n zero matrix by Z n, and the zero vector by θ n An inner product
More information5 Banach Algebras. 5.1 Invertibility and the Spectrum. Robert Oeckl FA NOTES 5 19/05/2010 1
Robert Oeckl FA NOTES 5 19/05/2010 1 5 Banach Algebras 5.1 Invertibility and the Spectrum Suppose X is a Banach space. Then we are often interested in (continuous) operators on this space, i.e, elements
More informationUCLA Chemical Engineering. Process & Control Systems Engineering Laboratory
Constrained Innite-Time Nonlinear Quadratic Optimal Control V. Manousiouthakis D. Chmielewski Chemical Engineering Department UCLA 1998 AIChE Annual Meeting Outline Unconstrained Innite-Time Nonlinear
More informationSemi-strongly asymptotically non-expansive mappings and their applications on xed point theory
Hacettepe Journal of Mathematics and Statistics Volume 46 (4) (2017), 613 620 Semi-strongly asymptotically non-expansive mappings and their applications on xed point theory Chris Lennard and Veysel Nezir
More information12 CHAPTER 1. PRELIMINARIES Lemma 1.3 (Cauchy-Schwarz inequality) Let (; ) be an inner product in < n. Then for all x; y 2 < n we have j(x; y)j (x; x)
1.4. INNER PRODUCTS,VECTOR NORMS, AND MATRIX NORMS 11 The estimate ^ is unbiased, but E(^ 2 ) = n?1 n 2 and is thus biased. An unbiased estimate is ^ 2 = 1 (x i? ^) 2 : n? 1 In x?? we show that the linear
More informationGeneralization theory
Generalization theory Daniel Hsu Columbia TRIPODS Bootcamp 1 Motivation 2 Support vector machines X = R d, Y = { 1, +1}. Return solution ŵ R d to following optimization problem: λ min w R d 2 w 2 2 + 1
More informationsatisfying ( i ; j ) = ij Here ij = if i = j and 0 otherwise The idea to use lattices is the following Suppose we are given a lattice L and a point ~x
Dual Vectors and Lower Bounds for the Nearest Lattice Point Problem Johan Hastad* MIT Abstract: We prove that given a point ~z outside a given lattice L then there is a dual vector which gives a fairly
More informationProjection Theorem 1
Projection Theorem 1 Cauchy-Schwarz Inequality Lemma. (Cauchy-Schwarz Inequality) For all x, y in an inner product space, [ xy, ] x y. Equality holds if and only if x y or y θ. Proof. If y θ, the inequality
More informationj=1 [We will show that the triangle inequality holds for each p-norm in Chapter 3 Section 6.] The 1-norm is A F = tr(a H A).
Math 344 Lecture #19 3.5 Normed Linear Spaces Definition 3.5.1. A seminorm on a vector space V over F is a map : V R that for all x, y V and for all α F satisfies (i) x 0 (positivity), (ii) αx = α x (scale
More informationA SYNOPSIS OF HILBERT SPACE THEORY
A SYNOPSIS OF HILBERT SPACE THEORY Below is a summary of Hilbert space theory that you find in more detail in any book on functional analysis, like the one Akhiezer and Glazman, the one by Kreiszig or
More informationConvex Analysis and Optimization Chapter 2 Solutions
Convex Analysis and Optimization Chapter 2 Solutions Dimitri P. Bertsekas with Angelia Nedić and Asuman E. Ozdaglar Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com
More informationINVERSE FUNCTION THEOREM and SURFACES IN R n
INVERSE FUNCTION THEOREM and SURFACES IN R n Let f C k (U; R n ), with U R n open. Assume df(a) GL(R n ), where a U. The Inverse Function Theorem says there is an open neighborhood V U of a in R n so that
More informationquantitative information on the error caused by using the solution of the linear problem to describe the response of the elastic material on a corner
Quantitative Justication of Linearization in Nonlinear Hencky Material Problems 1 Weimin Han and Hong-ci Huang 3 Abstract. The classical linear elasticity theory is based on the assumption that the size
More informationConverse Lyapunov Functions for Inclusions 2 Basic denitions Given a set A, A stands for the closure of A, A stands for the interior set of A, coa sta
A smooth Lyapunov function from a class-kl estimate involving two positive semidenite functions Andrew R. Teel y ECE Dept. University of California Santa Barbara, CA 93106 teel@ece.ucsb.edu Laurent Praly
More informationPlan of Class 4. Radial Basis Functions with moving centers. Projection Pursuit Regression and ridge. Principal Component Analysis: basic ideas
Plan of Class 4 Radial Basis Functions with moving centers Multilayer Perceptrons Projection Pursuit Regression and ridge functions approximation Principal Component Analysis: basic ideas Radial Basis
More informationIntroduction to Empirical Processes and Semiparametric Inference Lecture 13: Entropy Calculations
Introduction to Empirical Processes and Semiparametric Inference Lecture 13: Entropy Calculations Michael R. Kosorok, Ph.D. Professor and Chair of Biostatistics Professor of Statistics and Operations Research
More information2 Garrett: `A Good Spectral Theorem' 1. von Neumann algebras, density theorem The commutant of a subring S of a ring R is S 0 = fr 2 R : rs = sr; 8s 2
1 A Good Spectral Theorem c1996, Paul Garrett, garrett@math.umn.edu version February 12, 1996 1 Measurable Hilbert bundles Measurable Banach bundles Direct integrals of Hilbert spaces Trivializing Hilbert
More informationNonlinear Control. Nonlinear Control Lecture # 8 Time Varying and Perturbed Systems
Nonlinear Control Lecture # 8 Time Varying and Perturbed Systems Time-varying Systems ẋ = f(t,x) f(t,x) is piecewise continuous in t and locally Lipschitz in x for all t 0 and all x D, (0 D). The origin
More informationOptimality Conditions for Nonsmooth Convex Optimization
Optimality Conditions for Nonsmooth Convex Optimization Sangkyun Lee Oct 22, 2014 Let us consider a convex function f : R n R, where R is the extended real field, R := R {, + }, which is proper (f never
More informationFunctional Analysis. Franck Sueur Metric spaces Definitions Completeness Compactness Separability...
Functional Analysis Franck Sueur 2018-2019 Contents 1 Metric spaces 1 1.1 Definitions........................................ 1 1.2 Completeness...................................... 3 1.3 Compactness......................................
More informationON TRIVIAL GRADIENT YOUNG MEASURES BAISHENG YAN Abstract. We give a condition on a closed set K of real nm matrices which ensures that any W 1 p -grad
ON TRIVIAL GRAIENT YOUNG MEASURES BAISHENG YAN Abstract. We give a condition on a closed set K of real nm matrices which ensures that any W 1 p -gradient Young measure supported on K must be trivial the
More informationBasic Theorems about Independence, Spanning, Basis, and Dimension
Basic Theorems about Independence, Spanning, Basis, and Dimension Theorem 1 If v 1 ; : : : ; v m span V; then every set of vectors in V with more than m vectors must be linearly dependent. Proof. Let w
More information1. Subspaces A subset M of Hilbert space H is a subspace of it is closed under the operation of forming linear combinations;i.e.,
Abstract Hilbert Space Results We have learned a little about the Hilbert spaces L U and and we have at least defined H 1 U and the scale of Hilbert spaces H p U. Now we are going to develop additional
More informationDIFFERENTIAL OPERATORS ON DOMAINS WITH CONICAL POINTS: PRECISE UNIFORM REGULARITY ESTIMATES
Dedicated to Monique Dauge on her 60 th birthday DIFFERENTIAL OPERATORS ON DOMAINS WITH CONICAL POINTS: PRECISE UNIFORM REGULARITY ESTIMATES CONSTANTIN BÄCU TÄ, HENGGUANG LI and VICTOR NISTOR Communicated
More informationVariational Formulations
Chapter 2 Variational Formulations In this chapter we will derive a variational (or weak) formulation of the elliptic boundary value problem (1.4). We will discuss all fundamental theoretical results that
More informationUNBOUNDED OPERATORS ON HILBERT SPACES. Let X and Y be normed linear spaces, and suppose A : X Y is a linear map.
UNBOUNDED OPERATORS ON HILBERT SPACES EFTON PARK Let X and Y be normed linear spaces, and suppose A : X Y is a linear map. Define { } Ax A op = sup x : x 0 = { Ax : x 1} = { Ax : x = 1} If A
More informationA NOTE ON LINEAR FUNCTIONAL NORMS
A NOTE ON LINEAR FUNCTIONAL NORMS YIFEI PAN AND MEI WANG Abstract. For a vector u in a normed linear space, Hahn-Banach Theorem provides the existence of a linear functional f, f(u) = u such that f = 1.
More informationLocally convex spaces, the hyperplane separation theorem, and the Krein-Milman theorem
56 Chapter 7 Locally convex spaces, the hyperplane separation theorem, and the Krein-Milman theorem Recall that C(X) is not a normed linear space when X is not compact. On the other hand we could use semi
More informationApplied Analysis (APPM 5440): Final exam 1:30pm 4:00pm, Dec. 14, Closed books.
Applied Analysis APPM 44: Final exam 1:3pm 4:pm, Dec. 14, 29. Closed books. Problem 1: 2p Set I = [, 1]. Prove that there is a continuous function u on I such that 1 ux 1 x sin ut 2 dt = cosx, x I. Define
More informationProblem Set 5: Solutions Math 201A: Fall 2016
Problem Set 5: s Math 21A: Fall 216 Problem 1. Define f : [1, ) [1, ) by f(x) = x + 1/x. Show that f(x) f(y) < x y for all x, y [1, ) with x y, but f has no fixed point. Why doesn t this example contradict
More informationFrom Fractional Brownian Motion to Multifractional Brownian Motion
From Fractional Brownian Motion to Multifractional Brownian Motion Antoine Ayache USTL (Lille) Antoine.Ayache@math.univ-lille1.fr Cassino December 2010 A.Ayache (USTL) From FBM to MBM Cassino December
More information6. Duals of L p spaces
6 Duals of L p spaces This section deals with the problem if identifying the duals of L p spaces, p [1, ) There are essentially two cases of this problem: (i) p = 1; (ii) 1 < p < The major difference between
More informationi=1 β i,i.e. = β 1 x β x β 1 1 xβ d
66 2. Every family of seminorms on a vector space containing a norm induces ahausdorff locally convex topology. 3. Given an open subset Ω of R d with the euclidean topology, the space C(Ω) of real valued
More informationBoosting with Early Stopping: Convergence and Consistency
Boosting with Early Stopping: Convergence and Consistency Tong Zhang Bin Yu Abstract Boosting is one of the most significant advances in machine learning for classification and regression. In its original
More informationChapter 1. Optimality Conditions: Unconstrained Optimization. 1.1 Differentiable Problems
Chapter 1 Optimality Conditions: Unconstrained Optimization 1.1 Differentiable Problems Consider the problem of minimizing the function f : R n R where f is twice continuously differentiable on R n : P
More informationCHARACTERISTIC POINTS, RECTIFIABILITY AND PERIMETER MEASURE ON STRATIFIED GROUPS
CHARACTERISTIC POINTS, RECTIFIABILITY AND PERIMETER MEASURE ON STRATIFIED GROUPS VALENTINO MAGNANI Abstract. We establish an explicit formula between the perimeter measure of an open set E with C 1 boundary
More informationVector Spaces ปร ภ ม เวกเตอร
Vector Spaces ปร ภ ม เวกเตอร 5.1 Real Vector Spaces ปร ภ ม เวกเตอร ของจ านวนจร ง Vector Space Axioms (1/2) Let V be an arbitrary nonempty set of objects on which two operations are defined, addition and
More informationMARKOV CHAINS: STATIONARY DISTRIBUTIONS AND FUNCTIONS ON STATE SPACES. Contents
MARKOV CHAINS: STATIONARY DISTRIBUTIONS AND FUNCTIONS ON STATE SPACES JAMES READY Abstract. In this paper, we rst introduce the concepts of Markov Chains and their stationary distributions. We then discuss
More informationLinear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) 1.1 The Formal Denition of a Vector Space
Linear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) Contents 1 Vector Spaces 1 1.1 The Formal Denition of a Vector Space.................................. 1 1.2 Subspaces...................................................
More informationComputation Of Asymptotic Distribution. For Semiparametric GMM Estimators. Hidehiko Ichimura. Graduate School of Public Policy
Computation Of Asymptotic Distribution For Semiparametric GMM Estimators Hidehiko Ichimura Graduate School of Public Policy and Graduate School of Economics University of Tokyo A Conference in honor of
More informationr( = f 2 L 2 (1.2) iku)! 0 as r!1: (1.3) It was shown in book [7] that if f is assumed to be the restriction of a function in C
Inverse Obstacle Problem: Local Uniqueness for Rougher Obstacles and the Identication of A Ball Changmei Liu Department of Mathematics University of North Carolina Chapel Hill, NC 27599 December, 1995
More informationLecture 2: Review of Prerequisites. Table of contents
Math 348 Fall 217 Lecture 2: Review of Prerequisites Disclaimer. As we have a textbook, this lecture note is for guidance and supplement only. It should not be relied on when preparing for exams. In this
More informationHonours Analysis III
Honours Analysis III Math 354 Prof. Dmitry Jacobson Notes Taken By: R. Gibson Fall 2010 1 Contents 1 Overview 3 1.1 p-adic Distance............................................ 4 2 Introduction 5 2.1 Normed
More informationReproducing Kernel Hilbert Spaces Class 03, 15 February 2006 Andrea Caponnetto
Reproducing Kernel Hilbert Spaces 9.520 Class 03, 15 February 2006 Andrea Caponnetto About this class Goal To introduce a particularly useful family of hypothesis spaces called Reproducing Kernel Hilbert
More informationMeasure and Integration: Solutions of CW2
Measure and Integration: s of CW2 Fall 206 [G. Holzegel] December 9, 206 Problem of Sheet 5 a) Left (f n ) and (g n ) be sequences of integrable functions with f n (x) f (x) and g n (x) g (x) for almost
More informationImportance Sampling for Minibatches
Importance Sampling for Minibatches Dominik Csiba School of Mathematics University of Edinburgh 07.09.2016, Birmingham Dominik Csiba (University of Edinburgh) Importance Sampling for Minibatches 07.09.2016,
More informationIn English, this means that if we travel on a straight line between any two points in C, then we never leave C.
Convex sets In this section, we will be introduced to some of the mathematical fundamentals of convex sets. In order to motivate some of the definitions, we will look at the closest point problem from
More informationA Proof of the EOQ Formula Using Quasi-Variational. Inequalities. March 19, Abstract
A Proof of the EOQ Formula Using Quasi-Variational Inequalities Dir Beyer y and Suresh P. Sethi z March, 8 Abstract In this paper, we use quasi-variational inequalities to provide a rigorous proof of the
More informationPointwise convergence rate for nonlinear conservation. Eitan Tadmor and Tao Tang
Pointwise convergence rate for nonlinear conservation laws Eitan Tadmor and Tao Tang Abstract. We introduce a new method to obtain pointwise error estimates for vanishing viscosity and nite dierence approximations
More informationSome Results on b-orthogonality in 2-Normed Linear Spaces
Int. Journal of Math. Analysis, Vol. 1, 2007, no. 14, 681-687 Some Results on b-orthogonality in 2-Normed Linear Spaces H. Mazaheri and S. Golestani Nezhad Department of Mathematics Yazd University, Yazd,
More informationAlgorithms for Nonsmooth Optimization
Algorithms for Nonsmooth Optimization Frank E. Curtis, Lehigh University presented at Center for Optimization and Statistical Learning, Northwestern University 2 March 2018 Algorithms for Nonsmooth Optimization
More informationMA651 Topology. Lecture 9. Compactness 2.
MA651 Topology. Lecture 9. Compactness 2. This text is based on the following books: Topology by James Dugundgji Fundamental concepts of topology by Peter O Neil Elements of Mathematics: General Topology
More informationLecture # 3 Orthogonal Matrices and Matrix Norms. We repeat the definition an orthogonal set and orthornormal set.
Lecture # 3 Orthogonal Matrices and Matrix Norms We repeat the definition an orthogonal set and orthornormal set. Definition A set of k vectors {u, u 2,..., u k }, where each u i R n, is said to be an
More informationElements of Convex Optimization Theory
Elements of Convex Optimization Theory Costis Skiadas August 2015 This is a revised and extended version of Appendix A of Skiadas (2009), providing a self-contained overview of elements of convex optimization
More informationConvex Optimization Theory. Chapter 5 Exercises and Solutions: Extended Version
Convex Optimization Theory Chapter 5 Exercises and Solutions: Extended Version Dimitri P. Bertsekas Massachusetts Institute of Technology Athena Scientific, Belmont, Massachusetts http://www.athenasc.com
More informationTransductive Experiment Design
Appearing in NIPS 2005 workshop Foundations of Active Learning, Whistler, Canada, December, 2005. Transductive Experiment Design Kai Yu, Jinbo Bi, Volker Tresp Siemens AG 81739 Munich, Germany Abstract
More informationNew concepts: Span of a vector set, matrix column space (range) Linearly dependent set of vectors Matrix null space
Lesson 6: Linear independence, matrix column space and null space New concepts: Span of a vector set, matrix column space (range) Linearly dependent set of vectors Matrix null space Two linear systems:
More informationMath 209B Homework 2
Math 29B Homework 2 Edward Burkard Note: All vector spaces are over the field F = R or C 4.6. Two Compactness Theorems. 4. Point Set Topology Exercise 6 The product of countably many sequentally compact
More informationUniversity of Missouri. In Partial Fulllment LINDSEY M. WOODLAND MAY 2015
Frames and applications: Distribution of frame coecients, integer frames and phase retrieval A Dissertation presented to the Faculty of the Graduate School University of Missouri In Partial Fulllment of
More informationLecture 11 Hyperbolicity.
Lecture 11 Hyperbolicity. 1 C 0 linearization near a hyperbolic point 2 invariant manifolds Hyperbolic linear maps. Let E be a Banach space. A linear map A : E E is called hyperbolic if we can find closed
More informationAn introduction to some aspects of functional analysis
An introduction to some aspects of functional analysis Stephen Semmes Rice University Abstract These informal notes deal with some very basic objects in functional analysis, including norms and seminorms
More informationTopics in Mathematical Economics. Atsushi Kajii Kyoto University
Topics in Mathematical Economics Atsushi Kajii Kyoto University 26 June 2018 2 Contents 1 Preliminary Mathematics 5 1.1 Topology.................................. 5 1.2 Linear Algebra..............................
More informationClass 2 & 3 Overfitting & Regularization
Class 2 & 3 Overfitting & Regularization Carlo Ciliberto Department of Computer Science, UCL October 18, 2017 Last Class The goal of Statistical Learning Theory is to find a good estimator f n : X Y, approximating
More informationON THE REGULARITY OF SAMPLE PATHS OF SUB-ELLIPTIC DIFFUSIONS ON MANIFOLDS
Bendikov, A. and Saloff-Coste, L. Osaka J. Math. 4 (5), 677 7 ON THE REGULARITY OF SAMPLE PATHS OF SUB-ELLIPTIC DIFFUSIONS ON MANIFOLDS ALEXANDER BENDIKOV and LAURENT SALOFF-COSTE (Received March 4, 4)
More informationConditional Value-at-Risk (CVaR) Norm: Stochastic Case
Conditional Value-at-Risk (CVaR) Norm: Stochastic Case Alexander Mafusalov, Stan Uryasev RESEARCH REPORT 03-5 Risk Management and Financial Engineering Lab Department of Industrial and Systems Engineering
More informationOn the Intrinsic Differentiability Theorem of Gromov-Schoen
On the Intrinsic Differentiability Theorem of Gromov-Schoen Georgios Daskalopoulos Brown University daskal@math.brown.edu Chikako Mese 2 Johns Hopkins University cmese@math.jhu.edu Abstract In this note,
More informationPreparatory Material for the European Intensive Program in Bydgoszcz 2011 Analytical and computer assisted methods in mathematical models
Preparatory Material for the European Intensive Program in Bydgoszcz 2011 Analytical and computer assisted methods in mathematical models September 4{18 Basics on the Lebesgue integral and the divergence
More informationViscosity Iterative Approximating the Common Fixed Points of Non-expansive Semigroups in Banach Spaces
Viscosity Iterative Approximating the Common Fixed Points of Non-expansive Semigroups in Banach Spaces YUAN-HENG WANG Zhejiang Normal University Department of Mathematics Yingbing Road 688, 321004 Jinhua
More information1 Cheeger differentiation
1 Cheeger differentiation after J. Cheeger [1] and S. Keith [3] A summary written by John Mackay Abstract We construct a measurable differentiable structure on any metric measure space that is doubling
More informationarxiv: v1 [math.oc] 21 Mar 2015
Convex KKM maps, monotone operators and Minty variational inequalities arxiv:1503.06363v1 [math.oc] 21 Mar 2015 Marc Lassonde Université des Antilles, 97159 Pointe à Pitre, France E-mail: marc.lassonde@univ-ag.fr
More informationFractional Roman Domination
Chapter 6 Fractional Roman Domination It is important to discuss minimality of Roman domination functions before we get into the details of fractional version of Roman domination. Minimality of domination
More informationi. v = 0 if and only if v 0. iii. v + w v + w. (This is the Triangle Inequality.)
Definition 5.5.1. A (real) normed vector space is a real vector space V, equipped with a function called a norm, denoted by, provided that for all v and w in V and for all α R the real number v 0, and
More informationMachine Learning. Support Vector Machines. Fabio Vandin November 20, 2017
Machine Learning Support Vector Machines Fabio Vandin November 20, 2017 1 Classification and Margin Consider a classification problem with two classes: instance set X = R d label set Y = { 1, 1}. Training
More informationNORMS ON SPACE OF MATRICES
NORMS ON SPACE OF MATRICES. Operator Norms on Space of linear maps Let A be an n n real matrix and x 0 be a vector in R n. We would like to use the Picard iteration method to solve for the following system
More informationLebesgue Measure on R n
8 CHAPTER 2 Lebesgue Measure on R n Our goal is to construct a notion of the volume, or Lebesgue measure, of rather general subsets of R n that reduces to the usual volume of elementary geometrical sets
More informationFUNCTIONAL ANALYSIS-NORMED SPACE
MAT641- MSC Mathematics, MNIT Jaipur FUNCTIONAL ANALYSIS-NORMED SPACE DR. RITU AGARWAL MALAVIYA NATIONAL INSTITUTE OF TECHNOLOGY JAIPUR 1. Normed space Norm generalizes the concept of length in an arbitrary
More informationSobolev Spaces. Chapter 10
Chapter 1 Sobolev Spaces We now define spaces H 1,p (R n ), known as Sobolev spaces. For u to belong to H 1,p (R n ), we require that u L p (R n ) and that u have weak derivatives of first order in L p
More informationVector Space Basics. 1 Abstract Vector Spaces. 1. (commutativity of vector addition) u + v = v + u. 2. (associativity of vector addition)
Vector Space Basics (Remark: these notes are highly formal and may be a useful reference to some students however I am also posting Ray Heitmann's notes to Canvas for students interested in a direct computational
More informationPavel Dvurechensky Alexander Gasnikov Alexander Tiurin. July 26, 2017
Randomized Similar Triangles Method: A Unifying Framework for Accelerated Randomized Optimization Methods Coordinate Descent, Directional Search, Derivative-Free Method) Pavel Dvurechensky Alexander Gasnikov
More informationChapter 2: Preliminaries and elements of convex analysis
Chapter 2: Preliminaries and elements of convex analysis Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Website: http://home.deib.polimi.it/amaldi/opt-14-15.shtml Academic year 2014-15
More informationLECTURE 16: LIE GROUPS AND THEIR LIE ALGEBRAS. 1. Lie groups
LECTURE 16: LIE GROUPS AND THEIR LIE ALGEBRAS 1. Lie groups A Lie group is a special smooth manifold on which there is a group structure, and moreover, the two structures are compatible. Lie groups are
More informationValue and Policy Iteration
Chapter 7 Value and Policy Iteration 1 For infinite horizon problems, we need to replace our basic computational tool, the DP algorithm, which we used to compute the optimal cost and policy for finite
More informationKernel Methods. Jean-Philippe Vert Last update: Jan Jean-Philippe Vert (Mines ParisTech) 1 / 444
Kernel Methods Jean-Philippe Vert Jean-Philippe.Vert@mines.org Last update: Jan 2015 Jean-Philippe Vert (Mines ParisTech) 1 / 444 What we know how to solve Jean-Philippe Vert (Mines ParisTech) 2 / 444
More informationIntroduction to Statistical Learning Theory
Introduction to Statistical Learning Theory Definition Reminder: We are given m samples {(x i, y i )} m i=1 Dm and a hypothesis space H and we wish to return h H minimizing L D (h) = E[l(h(x), y)]. Problem
More information