Lecture Introduction. 2 Examples of Measure Concentration. 3 The Johnson-Lindenstrauss Lemma. CS-621 Theory Gems November 28, 2012
|
|
- Griselda Tucker
- 5 years ago
- Views:
Transcription
1 CS-6 Theory Gems November 8, 0 Lecture Lecturer: Alesaner Mąry Scribes: Alhussein Fawzi, Dorina Thanou Introuction Toay, we will briefly iscuss an important technique in probability theory measure concentration Roughly speaing, measure concentration correspons to exploiting a phenomenon that some functions of ranom variables are highly concentrate aroun their expectation/meian The main example that will be of our interest here is Johnson-Linenstrauss JL) lemma The JL lemma is a very powerful tool for imensionality reuction in high-imensional Eucliean spaces an it is wiely use to alleviate the curse of imensionality that occurs in applications where one nees to eal with high-imensional ata Examples of Measure Concentration Probably the most well-nown example of measure concentration result states that the sum of inepenent ranom variables is tightly concentrate aroun its expectation/meian In particular, if X, X,, X n are inepenent an ientically istribute ranom variables ii) with each X i taing a value X i {, } with equal probability, the celebrate Chernoff boun states that their sum X = n i= X i is highly concentrate aroun its expectation Specifically, the probability that X > t is exponentially ecaying with t, ie, X > t) < e t n ) Note that expectation of X is just zero) Although this result is the most well-nown one an it alreay has plethora of applications, it can actually be seen as a special case of a more general measure concentration phenomena To this en, let us focus our attention on general real functions on hypercube an say that a function f : {, } n R is L-Lipschitz, for some L > 0, with respect to l metric) iff, for all x, y {, } n, fx) fy) L x y ) One can view the L-Lipschitz conition as a quantifie version of uniform continuity of f) Now, one can show that for any -Lipschitz function of n ranom variables X, X,, X n that are ii an are + an with equal probability, an analogous to ) concentration aroun the meian µ of f occurs Namely, we have fx,, X n ) > µ + t) < e t n 3) One can get a result for arbitrary Lipschitz constant L just by scaling) As the sum function is clearly -Lipschitz, one can see that Chernoff boun is inee a consequence of this more general statement 3 The Johnson-Linenstrauss Lemma The main example of measure concentration phenomenon that we want to focus on toay is capture by Johnson-Linenstrauss JL) lemma an correspons to the behavior of ranom vectors on a highimensional unit sphere Roughly speaing, the Johnson-Linenstrauss lemma tells us that the l -istance of high-imensional vectors is well preserve uner ranom projection to a much) lower imension
2 Lemma Johnson-Linenstrauss lemma) Consier a set of n vectors x i R an a ranom - imensional subspace of R Let y i be the projection of each x i on that subspace For any ε > 0, if = Ωɛ log n) then with probability at least n, ɛ) xi x j y i y j + ɛ) xi x j, i, j 4) In the light of this lemma, if we have some high-imensional ata whose ey characteristic we are intereste in is capture by l -istance, then we can achieve even an exponential compression of this ata s imension at the price of introucing only + ε) error note that the is just a normalizing scaling factor) It turns out that there is a lot of scenarios especially, in statistics an machine learning) where this technique is applicable an allows one to lift the curse of imensionality Namely, in a lot of applications, very) high-imensional ata arises naturally an this in of compression often calle imensionality reuction provies a powerful tool for ealing with the computational cost of processing such ata 3 Ranom Subspaces Before proceeing to the proof of this lemma, we first nee to mae the notion of ranom subspace precise To this en, let us start by efining what we mean by a ranom unit vector x S, where S is the -imensional unit sphere We will view such a vector as a result of a generation proceure in which, first, we sample each of its coorinates inepenently from a Gaussian istribution N 0, ) that has zero mean an stanar eviation one; an then normalize it to mae its norm equal to Note that one of the important an esirable properties of this efinition is that the resulting probability measure on the sphere is rotationally invariant) Once we efine our notion of ranom unit vector, ie, we efine our probability measure on the sphere, we can procee to efining what we mean by a ranom subspace of imension Again, we will o this by specifying the ranom process that generates it This process is as follows: Choose a ranom unit vector an mae it the first basis vector v of the subspace For the next rouns repeat the following: pic a ranom unit vector, subtract from it its projection on the subspace spanne by the previously chosen vectors v,, v i, an normalize it to form the next basis vector v i Clearly, after this proceure is finishe we en up with an orthonormal basis v,, v that spans the esire ranom) subspace of imension Note that the above proceure is nothing else than Gram-Schmit orthogonalization applie to a set of ranom unit vectors {v,, v }) Also, one can see that uner this efinition the projection y i of a ata point x i onto such a ranom subspace can be written in a matrix form as v v v x i v v v x i y i y i = v v v y i }{{} x i V Here, each row of the projection matrix V correspons to one ranom basis vector v i It is easy to see that this ranomly chosen unit vector is not in the span of the vectors v,, v i with probability
3 3 Proof of the JL Lemma Now that we have efine what a ranom vector an what a ranom subspace is, we are reay to prove the Johnson-Linenstrauss lemma As a first step, we show that this lemma follows from a simpler statement that just focuses on the norm of the projection of a fixe vector x in imensions onto a ranom -imensional subspace Lemma Let x be an arbitrary vector in R an z R be its projection onto a ranom -imensional subspace Then, for any ε > 0, as long as = Ωɛ log n), we have z x ɛ, with probability exceeing n 3 It is not har to see that once we prove Lemma, the Johnson-Linenstrauss lemma follows easily Inee, by applying the above lemma with x = x i x j, for any fixe i an j, we get [ z i,j ] x i x j ɛ n 3, where z i,j is the projection of x i x j on the ranom subspace Since the projection is a linear map, we have z i,j = y i y j So, applying a union boun to the previous inequality, over all On ) pairs i, j), we get that [ y i y j ] i j, x i x j ɛ nn ) n 3 n, which can be easily seen to be equivalent to the statement of Johnson-Linenstrauss lemma Hence, from now on we focus on proving Lemma Observe that by scaling, it suffices to prove this lemma for the case of x being a unit vector) To mae our tas easier, we want to first invert our perspective Namely, instea of looing at the norm of a projection of an arbitrary vector onto a ranom -imensional subspace, we prefer to loo at the norm of a projection of a ranom vector on a fixe -imensional subspace that correspons to the first coorinates of that vector It is not har to see that these two views are completely equivalent To this en, note that we can always rotate the space in such a way that the ranom -imensional subspace we chosen is just the projection onto the first coorinates Formally, let U enote the unitary matrix whose first rows are equal to vectors v i s that form the basis of the ranom subspace we have chosen) an where the remaining rows are chosen arbitrarily to form an orthonormal basis of the orthogonal complement of our subspace Then, we have that z i = v i ) T x = U v i ) T U x), for any i, as U = U T is a unitary matrix too an thus satisfies U ) T U = I Since U v i is equal to the i-th stanar basis vector e i an U x is a ranom vector as it correspons to a ranom rotation of a fixe vector), it is inee vali to see z as the projection of a ranom vector onto the subspace spanne by its first coorinates Thans to the above simplification of the perspective, our goal now is to stuy how the norm of the first coorinates of a ranom vector of unit norm) concentrates aroun a particular value To this en, note that if z = z,, z ) = z, z +,, z ) is a ranom unit vector then clearly we have E i= z i ) = 3
4 Since each z i s are ientically istribute, we obtain ) E = i= z i Thus inee l -norm of the first coorinates of a ranom vector has the esire expectation However, to prove Lemma, we also nee to stuy how this norm is concentrate aroun its expectation We will not o this toay Instea, just to give a flavor of involve techniques, we prove here a simpler result that bouns the concentration of the corresponing norm for = Specifically, we show that the probability that z is larger than t is exponentially ecaying with t Lemma 3 Let z = z,, z ) be a ranom vector in S We have z > t ) ) exp t, for any 0 < t Proof The proof of this lemma is base on a simple geometric argument Let us fix some t > 0 As z a) b) Figure : Illustration of the proof in two imensions a) The caps corresponing to z > are mare with re color b) Pictorial argument justifying upperbouning the area of these two caps by the area of corresponing sphere of the same raius is a ranom vector from a -imensional unit sphere S, we can see that the probability of choosing z with z > t is exactly the ratio of the area of two -imensional caps of raius R cup = t to the total area of a unit -imensional sphere See Figure a) that represents the situation in two imensions, ie, the case of = ) We can upperboun the area of these two caps by the area of a whole sphere of the same raius see Figure b)) As the area of a -imensional sphere SR) of raius R has to be a function of the form C R, where C is some coefficient epening on but not on R, we have z > t ) areasr cup)) areas ) Using the fact that x/n) n exp x), we conclue that z > t ) exp t /), = C Rcup ) C = t t 4
5 whenever 0 < t, as esire It is interesting to note that by applying Lemma 3 with t = Ω log n), we get that the probability log n that z excees is boune by This tells us that in high imensions almost all the vectors n O) on the unit sphere are close to being orthogonal Inee, thans to the rotation invariance of the scalar prouct, we can always tae one of the vectors to have its first coorinate be equal to an have all the remaining coorinates equal to zero Then, the scalar prouct of a ranom unit vector z with this vector is equal to z In high imensions, the quantity log n is very small, which gives a very small scalar prouct with high probability Unfortunately, as we alreay mentione, the bouns provie by the Lemma 3 are too wea to yiel the esire concentration of the norm of the projection of z on the first coorinates Therefore, we state without proof) a stronger version of Lemma 3 that allows one tae avantage of larger values of Lemma 4 Let z = z,, z ) = z, z +,, z ) be a ranom vector in S We have ) z > t e t Once we have this lemma, the proof of Lemma is straightforwar We just tae t = ε = 0ε ln n We then have ) z > ɛ n 3, which proves Lemma, an thus the Johnson-Linenstrauss lemma 33 Further Discussion As we presente it here, the JL lemma is not very practical This is so as our generation of the projection matrix V requires performing Gram-Schmit orthnormalization that is computationally quite expensive when n is large which is often the case) To circumvent this issue an mae JL lemma more practical, there was a lot of successful) wor on eveloping much more efficient constructions of the projection matrix V In these latest constructions, this matrix is generate via a very simple an easy to implement proceure that maes V have only few entries of each column being non-zero As a result, not only the whole construction is very efficient, but also the resulting matrix V is sparse ie, it has only small fraction of entries non-zero), which leas to computations of the projections of the input vectors being very efficient too All of these avancements mae JL lemma a truly practical tool Given the usefulness of JL lemma in applications that operate base on l -istance, it is natural to woner if similar results coul be achieve for other l p -istances Unfortunately, it seems that it is not the case an in fact for some of the istances eg, l -istance) there are strong lowerbouns on the possible imension reuction Also, it is nown that for l -istance, the imension reuction offere by JL lemma is essentially optimal) an 5
Least-Squares Regression on Sparse Spaces
Least-Squares Regression on Sparse Spaces Yuri Grinberg, Mahi Milani Far, Joelle Pineau School of Computer Science McGill University Montreal, Canaa {ygrinb,mmilan1,jpineau}@cs.mcgill.ca 1 Introuction
More informationTopic 7: Convergence of Random Variables
Topic 7: Convergence of Ranom Variables Course 003, 2016 Page 0 The Inference Problem So far, our starting point has been a given probability space (S, F, P). We now look at how to generate information
More informationLecture 6 : Dimensionality Reduction
CPS290: Algorithmic Founations of Data Science February 3, 207 Lecture 6 : Dimensionality Reuction Lecturer: Kamesh Munagala Scribe: Kamesh Munagala In this lecture, we will consier the roblem of maing
More informationMulti-View Clustering via Canonical Correlation Analysis
Technical Report TTI-TR-2008-5 Multi-View Clustering via Canonical Correlation Analysis Kamalika Chauhuri UC San Diego Sham M. Kakae Toyota Technological Institute at Chicago ABSTRACT Clustering ata in
More information7.1 Support Vector Machine
67577 Intro. to Machine Learning Fall semester, 006/7 Lecture 7: Support Vector Machines an Kernel Functions II Lecturer: Amnon Shashua Scribe: Amnon Shashua 7. Support Vector Machine We return now to
More informationA Unified Theorem on SDP Rank Reduction
A Unifie heorem on SDP Ran Reuction Anthony Man Cho So, Yinyu Ye, Jiawei Zhang November 9, 006 Abstract We consier the problem of fining a low ran approximate solution to a system of linear equations in
More informationLECTURE NOTES ON DVORETZKY S THEOREM
LECTURE NOTES ON DVORETZKY S THEOREM STEVEN HEILMAN Abstract. We present the first half of the paper [S]. In particular, the results below, unless otherwise state, shoul be attribute to G. Schechtman.
More informationRobust Forward Algorithms via PAC-Bayes and Laplace Distributions. ω Q. Pr (y(ω x) < 0) = Pr A k
A Proof of Lemma 2 B Proof of Lemma 3 Proof: Since the support of LL istributions is R, two such istributions are equivalent absolutely continuous with respect to each other an the ivergence is well-efine
More informationPDE Notes, Lecture #11
PDE Notes, Lecture # from Professor Jalal Shatah s Lectures Febuary 9th, 2009 Sobolev Spaces Recall that for u L loc we can efine the weak erivative Du by Du, φ := udφ φ C0 If v L loc such that Du, φ =
More informationLower bounds on Locality Sensitive Hashing
Lower bouns on Locality Sensitive Hashing Rajeev Motwani Assaf Naor Rina Panigrahy Abstract Given a metric space (X, X ), c 1, r > 0, an p, q [0, 1], a istribution over mappings H : X N is calle a (r,
More informationFunction Spaces. 1 Hilbert Spaces
Function Spaces A function space is a set of functions F that has some structure. Often a nonparametric regression function or classifier is chosen to lie in some function space, where the assume structure
More informationDatabase-friendly Random Projections
Database-frienly Ranom Projections Dimitris Achlioptas Microsoft ABSTRACT A classic result of Johnson an Linenstrauss asserts that any set of n points in -imensional Eucliean space can be embee into k-imensional
More informationSome Examples. Uniform motion. Poisson processes on the real line
Some Examples Our immeiate goal is to see some examples of Lévy processes, an/or infinitely-ivisible laws on. Uniform motion Choose an fix a nonranom an efine X := for all (1) Then, {X } is a [nonranom]
More informationMulti-View Clustering via Canonical Correlation Analysis
Keywors: multi-view learning, clustering, canonical correlation analysis Abstract Clustering ata in high-imensions is believe to be a har problem in general. A number of efficient clustering algorithms
More informationHigh-Dimensional p-norms
High-Dimensional p-norms Gérar Biau an Davi M. Mason Abstract Let X = X 1,...,X be a R -value ranom vector with i.i.. components, an let X p = j=1 X j p 1/p be its p-norm, for p > 0. The impact of letting
More informationAcute sets in Euclidean spaces
Acute sets in Eucliean spaces Viktor Harangi April, 011 Abstract A finite set H in R is calle an acute set if any angle etermine by three points of H is acute. We examine the maximal carinality α() of
More informationLecture 2 Lagrangian formulation of classical mechanics Mechanics
Lecture Lagrangian formulation of classical mechanics 70.00 Mechanics Principle of stationary action MATH-GA To specify a motion uniquely in classical mechanics, it suffices to give, at some time t 0,
More informationNecessary and Sufficient Conditions for Sketched Subspace Clustering
Necessary an Sufficient Conitions for Sketche Subspace Clustering Daniel Pimentel-Alarcón, Laura Balzano 2, Robert Nowak University of Wisconsin-Maison, 2 University of Michigan-Ann Arbor Abstract This
More informationConvergence of Random Walks
Chapter 16 Convergence of Ranom Walks This lecture examines the convergence of ranom walks to the Wiener process. This is very important both physically an statistically, an illustrates the utility of
More informationAgmon Kolmogorov Inequalities on l 2 (Z d )
Journal of Mathematics Research; Vol. 6, No. ; 04 ISSN 96-9795 E-ISSN 96-9809 Publishe by Canaian Center of Science an Eucation Agmon Kolmogorov Inequalities on l (Z ) Arman Sahovic Mathematics Department,
More informationGeneralized Tractability for Multivariate Problems
Generalize Tractability for Multivariate Problems Part II: Linear Tensor Prouct Problems, Linear Information, an Unrestricte Tractability Michael Gnewuch Department of Computer Science, University of Kiel,
More informationarxiv: v1 [cs.lg] 22 Mar 2014
CUR lgorithm with Incomplete Matrix Observation Rong Jin an Shenghuo Zhu Dept. of Computer Science an Engineering, Michigan State University, rongjin@msu.eu NEC Laboratories merica, Inc., zsh@nec-labs.com
More informationSeparation of Variables
Physics 342 Lecture 1 Separation of Variables Lecture 1 Physics 342 Quantum Mechanics I Monay, January 25th, 2010 There are three basic mathematical tools we nee, an then we can begin working on the physical
More informationu!i = a T u = 0. Then S satisfies
Deterministic Conitions for Subspace Ientifiability from Incomplete Sampling Daniel L Pimentel-Alarcón, Nigel Boston, Robert D Nowak University of Wisconsin-Maison Abstract Consier an r-imensional subspace
More informationAnalyzing Tensor Power Method Dynamics in Overcomplete Regime
Journal of Machine Learning Research 18 (2017) 1-40 Submitte 9/15; Revise 11/16; Publishe 4/17 Analyzing Tensor Power Metho Dynamics in Overcomplete Regime Animashree Ananumar Department of Electrical
More informationG4003 Advanced Mechanics 1. We already saw that if q is a cyclic variable, the associated conjugate momentum is conserved, L = const.
G4003 Avance Mechanics 1 The Noether theorem We alreay saw that if q is a cyclic variable, the associate conjugate momentum is conserve, q = 0 p q = const. (1) This is the simplest incarnation of Noether
More informationMulti-View Clustering via Canonical Correlation Analysis
Kamalika Chauhuri ITA, UC San Diego, 9500 Gilman Drive, La Jolla, CA Sham M. Kakae Karen Livescu Karthik Sriharan Toyota Technological Institute at Chicago, 6045 S. Kenwoo Ave., Chicago, IL kamalika@soe.ucs.eu
More informationAn Optimal Algorithm for Bandit and Zero-Order Convex Optimization with Two-Point Feedback
Journal of Machine Learning Research 8 07) - Submitte /6; Publishe 5/7 An Optimal Algorithm for Banit an Zero-Orer Convex Optimization with wo-point Feeback Oha Shamir Department of Computer Science an
More informationensembles When working with density operators, we can use this connection to define a generalized Bloch vector: v x Tr x, v y Tr y
Ph195a lecture notes, 1/3/01 Density operators for spin- 1 ensembles So far in our iscussion of spin- 1 systems, we have restricte our attention to the case of pure states an Hamiltonian evolution. Toay
More informationDarboux s theorem and symplectic geometry
Darboux s theorem an symplectic geometry Liang, Feng May 9, 2014 Abstract Symplectic geometry is a very important branch of ifferential geometry, it is a special case of poisson geometry, an coul also
More informationMulti-View Clustering via Canonical Correlation Analysis
Kamalika Chauhuri ITA, UC San Diego, 9500 Gilman Drive, La Jolla, CA Sham M. Kakae Karen Livescu Karthik Sriharan Toyota Technological Institute at Chicago, 6045 S. Kenwoo Ave., Chicago, IL kamalika@soe.ucs.eu
More informationProblem Sheet 2: Eigenvalues and eigenvectors and their use in solving linear ODEs
Problem Sheet 2: Eigenvalues an eigenvectors an their use in solving linear ODEs If you fin any typos/errors in this problem sheet please email jk28@icacuk The material in this problem sheet is not examinable
More informationTractability results for weighted Banach spaces of smooth functions
Tractability results for weighte Banach spaces of smooth functions Markus Weimar Mathematisches Institut, Universität Jena Ernst-Abbe-Platz 2, 07740 Jena, Germany email: markus.weimar@uni-jena.e March
More informationA Weak First Digit Law for a Class of Sequences
International Mathematical Forum, Vol. 11, 2016, no. 15, 67-702 HIKARI Lt, www.m-hikari.com http://x.oi.org/10.1288/imf.2016.6562 A Weak First Digit Law for a Class of Sequences M. A. Nyblom School of
More informationComputing Exact Confidence Coefficients of Simultaneous Confidence Intervals for Multinomial Proportions and their Functions
Working Paper 2013:5 Department of Statistics Computing Exact Confience Coefficients of Simultaneous Confience Intervals for Multinomial Proportions an their Functions Shaobo Jin Working Paper 2013:5
More informationIntroduction to the Vlasov-Poisson system
Introuction to the Vlasov-Poisson system Simone Calogero 1 The Vlasov equation Consier a particle with mass m > 0. Let x(t) R 3 enote the position of the particle at time t R an v(t) = ẋ(t) = x(t)/t its
More informationDiscrete Mathematics
Discrete Mathematics 309 (009) 86 869 Contents lists available at ScienceDirect Discrete Mathematics journal homepage: wwwelseviercom/locate/isc Profile vectors in the lattice of subspaces Dániel Gerbner
More informationLower Bounds for the Smoothed Number of Pareto optimal Solutions
Lower Bouns for the Smoothe Number of Pareto optimal Solutions Tobias Brunsch an Heiko Röglin Department of Computer Science, University of Bonn, Germany brunsch@cs.uni-bonn.e, heiko@roeglin.org Abstract.
More informationA Sketch of Menshikov s Theorem
A Sketch of Menshikov s Theorem Thomas Bao March 14, 2010 Abstract Let Λ be an infinite, locally finite oriente multi-graph with C Λ finite an strongly connecte, an let p
More informationThe Exact Form and General Integrating Factors
7 The Exact Form an General Integrating Factors In the previous chapters, we ve seen how separable an linear ifferential equations can be solve using methos for converting them to forms that can be easily
More informationarxiv: v4 [cs.ds] 7 Mar 2014
Analysis of Agglomerative Clustering Marcel R. Ackermann Johannes Blömer Daniel Kuntze Christian Sohler arxiv:101.697v [cs.ds] 7 Mar 01 Abstract The iameter k-clustering problem is the problem of partitioning
More informationarxiv: v2 [cs.ds] 11 May 2016
Optimizing Star-Convex Functions Jasper C.H. Lee Paul Valiant arxiv:5.04466v2 [cs.ds] May 206 Department of Computer Science Brown University {jasperchlee,paul_valiant}@brown.eu May 3, 206 Abstract We
More informationMath 1B, lecture 8: Integration by parts
Math B, lecture 8: Integration by parts Nathan Pflueger 23 September 2 Introuction Integration by parts, similarly to integration by substitution, reverses a well-known technique of ifferentiation an explores
More information1. Aufgabenblatt zur Vorlesung Probability Theory
24.10.17 1. Aufgabenblatt zur Vorlesung By (Ω, A, P ) we always enote the unerlying probability space, unless state otherwise. 1. Let r > 0, an efine f(x) = 1 [0, [ (x) exp( r x), x R. a) Show that p f
More informationEuler equations for multiple integrals
Euler equations for multiple integrals January 22, 2013 Contents 1 Reminer of multivariable calculus 2 1.1 Vector ifferentiation......................... 2 1.2 Matrix ifferentiation........................
More informationOn permutation-invariance of limit theorems
On permutation-invariance of limit theorems I. Beres an R. Tichy Abstract By a classical principle of probability theory, sufficiently thin subsequences of general sequences of ranom variables behave lie
More informationConcentration of Measure Inequalities for Compressive Toeplitz Matrices with Applications to Detection and System Identification
Concentration of Measure Inequalities for Compressive Toeplitz Matrices with Applications to Detection an System Ientification Borhan M Sananaji, Tyrone L Vincent, an Michael B Wakin Abstract In this paper,
More informationOn combinatorial approaches to compressed sensing
On combinatorial approaches to compresse sensing Abolreza Abolhosseini Moghaam an Hayer Raha Department of Electrical an Computer Engineering, Michigan State University, East Lansing, MI, U.S. Emails:{abolhos,raha}@msu.eu
More informationA PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks
A PAC-Bayesian Approach to Spectrally-Normalize Margin Bouns for Neural Networks Behnam Neyshabur, Srinah Bhojanapalli, Davi McAllester, Nathan Srebro Toyota Technological Institute at Chicago {bneyshabur,
More informationA Randomized Approximate Nearest Neighbors Algorithm - a short version
We present a ranomize algorithm for the approximate nearest neighbor problem in - imensional Eucliean space. Given N points {x } in R, the algorithm attempts to fin k nearest neighbors for each of x, where
More informationFLUCTUATIONS IN THE NUMBER OF POINTS ON SMOOTH PLANE CURVES OVER FINITE FIELDS. 1. Introduction
FLUCTUATIONS IN THE NUMBER OF POINTS ON SMOOTH PLANE CURVES OVER FINITE FIELDS ALINA BUCUR, CHANTAL DAVID, BROOKE FEIGON, MATILDE LALÍN 1 Introuction In this note, we stuy the fluctuations in the number
More informationOn the Surprising Behavior of Distance Metrics in High Dimensional Space
On the Surprising Behavior of Distance Metrics in High Dimensional Space Charu C. Aggarwal, Alexaner Hinneburg 2, an Daniel A. Keim 2 IBM T. J. Watson Research Center Yortown Heights, NY 0598, USA. charu@watson.ibm.com
More informationAnalysis IV, Assignment 4
Analysis IV, Assignment 4 Prof. John Toth Winter 23 Exercise Let f C () an perioic with f(x+2) f(x). Let a n f(t)e int t an (S N f)(x) N n N then f(x ) lim (S Nf)(x ). N a n e inx. If f is continuously
More informationAll s Well That Ends Well: Supplementary Proofs
All s Well That Ens Well: Guarantee Resolution of Simultaneous Rigi Boy Impact 1:1 All s Well That Ens Well: Supplementary Proofs This ocument complements the paper All s Well That Ens Well: Guarantee
More informationMA 2232 Lecture 08 - Review of Log and Exponential Functions and Exponential Growth
MA 2232 Lecture 08 - Review of Log an Exponential Functions an Exponential Growth Friay, February 2, 2018. Objectives: Review log an exponential functions, their erivative an integration formulas. Exponential
More informationMultivariate Statistics Random Projections and Johnson-Lindenstrauss Lemma
Multivariate Statistics Random Projections and Johnson-Lindenstrauss Lemma Suppose again we have n sample points x,..., x n R p. The data-point x i R p can be thought of as the i-th row X i of an n p-dimensional
More informationSINGULAR PERTURBATION AND STATIONARY SOLUTIONS OF PARABOLIC EQUATIONS IN GAUSS-SOBOLEV SPACES
Communications on Stochastic Analysis Vol. 2, No. 2 (28) 289-36 Serials Publications www.serialspublications.com SINGULAR PERTURBATION AND STATIONARY SOLUTIONS OF PARABOLIC EQUATIONS IN GAUSS-SOBOLEV SPACES
More informationThe Principle of Least Action
Chapter 7. The Principle of Least Action 7.1 Force Methos vs. Energy Methos We have so far stuie two istinct ways of analyzing physics problems: force methos, basically consisting of the application of
More informationSharp Thresholds. Zachary Hamaker. March 15, 2010
Sharp Threshols Zachary Hamaker March 15, 2010 Abstract The Kolmogorov Zero-One law states that for tail events on infinite-imensional probability spaces, the probability must be either zero or one. Behavior
More informationCalculus of Variations
Calculus of Variations Lagrangian formalism is the main tool of theoretical classical mechanics. Calculus of Variations is a part of Mathematics which Lagrangian formalism is base on. In this section,
More informationDelocalization of boundary states in disordered topological insulators
Journal of Physics A: Mathematical an Theoretical J. Phys. A: Math. Theor. 48 (05) FT0 (pp) oi:0.088/75-83/48//ft0 Fast Track Communication Delocalization of bounary states in isorere topological insulators
More informationChromatic number for a generalization of Cartesian product graphs
Chromatic number for a generalization of Cartesian prouct graphs Daniel Král Douglas B. West Abstract Let G be a class of graphs. The -fol gri over G, enote G, is the family of graphs obtaine from -imensional
More informationLATTICE-BASED D-OPTIMUM DESIGN FOR FOURIER REGRESSION
The Annals of Statistics 1997, Vol. 25, No. 6, 2313 2327 LATTICE-BASED D-OPTIMUM DESIGN FOR FOURIER REGRESSION By Eva Riccomagno, 1 Rainer Schwabe 2 an Henry P. Wynn 1 University of Warwick, Technische
More information19 Eigenvalues, Eigenvectors, Ordinary Differential Equations, and Control
19 Eigenvalues, Eigenvectors, Orinary Differential Equations, an Control This section introuces eigenvalues an eigenvectors of a matrix, an iscusses the role of the eigenvalues in etermining the behavior
More informationLow-Dimensional Lattice Basis Reduction Revisited (Extended Abstract)
Algorithmic Number Theory Proceeings of ANTS-VI (June 13 18, 2004, Burlington, U.S.A.) D. Buell (E.), vol.???? of Lecture Notes in Computer Science, pages?????? c Springer-Verlag (http://www.springer.e/comp/lncs/inex.html)
More informationSolutions to Math 41 Second Exam November 4, 2010
Solutions to Math 41 Secon Exam November 4, 2010 1. (13 points) Differentiate, using the metho of your choice. (a) p(t) = ln(sec t + tan t) + log 2 (2 + t) (4 points) Using the rule for the erivative of
More informationApproximate Constraint Satisfaction Requires Large LP Relaxations
Approximate Constraint Satisfaction Requires Large LP Relaxations oah Fleming April 19, 2018 Linear programming is a very powerful tool for attacking optimization problems. Techniques such as the ellipsoi
More informationInfluence of weight initialization on multilayer perceptron performance
Influence of weight initialization on multilayer perceptron performance M. Karouia (1,2) T. Denœux (1) R. Lengellé (1) (1) Université e Compiègne U.R.A. CNRS 817 Heuiasyc BP 649 - F-66 Compiègne ceex -
More informationLower Bounds for Local Monotonicity Reconstruction from Transitive-Closure Spanners
Lower Bouns for Local Monotonicity Reconstruction from Transitive-Closure Spanners Arnab Bhattacharyya Elena Grigorescu Mahav Jha Kyomin Jung Sofya Raskhonikova Davi P. Wooruff Abstract Given a irecte
More informationSurvey Sampling. 1 Design-based Inference. Kosuke Imai Department of Politics, Princeton University. February 19, 2013
Survey Sampling Kosuke Imai Department of Politics, Princeton University February 19, 2013 Survey sampling is one of the most commonly use ata collection methos for social scientists. We begin by escribing
More informationMonte Carlo Methods with Reduced Error
Monte Carlo Methos with Reuce Error As has been shown, the probable error in Monte Carlo algorithms when no information about the smoothness of the function is use is Dξ r N = c N. It is important for
More informationLeaving Randomness to Nature: d-dimensional Product Codes through the lens of Generalized-LDPC codes
Leaving Ranomness to Nature: -Dimensional Prouct Coes through the lens of Generalize-LDPC coes Tavor Baharav, Kannan Ramchanran Dept. of Electrical Engineering an Computer Sciences, U.C. Berkeley {tavorb,
More informationIntroduction to Markov Processes
Introuction to Markov Processes Connexions moule m44014 Zzis law Gustav) Meglicki, Jr Office of the VP for Information Technology Iniana University RCS: Section-2.tex,v 1.24 2012/12/21 18:03:08 gustav
More informationRank, Trace, Determinant, Transpose an Inverse of a Matrix Let A be an n n square matrix: A = a11 a1 a1n a1 a an a n1 a n a nn nn where is the jth col
Review of Linear Algebra { E18 Hanout Vectors an Their Inner Proucts Let X an Y be two vectors: an Their inner prouct is ene as X =[x1; ;x n ] T Y =[y1; ;y n ] T (X; Y ) = X T Y = x k y k k=1 where T an
More informationIterated Point-Line Configurations Grow Doubly-Exponentially
Iterate Point-Line Configurations Grow Doubly-Exponentially Joshua Cooper an Mark Walters July 9, 008 Abstract Begin with a set of four points in the real plane in general position. A to this collection
More informationSYSTEMS OF DIFFERENTIAL EQUATIONS, EULER S FORMULA. where L is some constant, usually called the Lipschitz constant. An example is
SYSTEMS OF DIFFERENTIAL EQUATIONS, EULER S FORMULA. Uniqueness for solutions of ifferential equations. We consier the system of ifferential equations given by x = v( x), () t with a given initial conition
More informationII. First variation of functionals
II. First variation of functionals The erivative of a function being zero is a necessary conition for the etremum of that function in orinary calculus. Let us now tackle the question of the equivalent
More informationMonotonicity for excited random walk in high dimensions
Monotonicity for excite ranom walk in high imensions Remco van er Hofsta Mark Holmes March, 2009 Abstract We prove that the rift θ, β) for excite ranom walk in imension is monotone in the excitement parameter
More informationAsymptotic estimates on the time derivative of entropy on a Riemannian manifold
Asymptotic estimates on the time erivative of entropy on a Riemannian manifol Arian P. C. Lim a, Dejun Luo b a Nanyang Technological University, athematics an athematics Eucation, Singapore b Key Lab of
More informationCollapsed Gibbs and Variational Methods for LDA. Example Collapsed MoG Sampling
Case Stuy : Document Retrieval Collapse Gibbs an Variational Methos for LDA Machine Learning/Statistics for Big Data CSE599C/STAT59, University of Washington Emily Fox 0 Emily Fox February 7 th, 0 Example
More informationQuantum mechanical approaches to the virial
Quantum mechanical approaches to the virial S.LeBohec Department of Physics an Astronomy, University of Utah, Salt Lae City, UT 84112, USA Date: June 30 th 2015 In this note, we approach the virial from
More informationTime-of-Arrival Estimation in Non-Line-Of-Sight Environments
2 Conference on Information Sciences an Systems, The Johns Hopkins University, March 2, 2 Time-of-Arrival Estimation in Non-Line-Of-Sight Environments Sinan Gezici, Hisashi Kobayashi an H. Vincent Poor
More informationarxiv: v1 [math.mg] 10 Apr 2018
ON THE VOLUME BOUND IN THE DVORETZKY ROGERS LEMMA FERENC FODOR, MÁRTON NASZÓDI, AND TAMÁS ZARNÓCZ arxiv:1804.03444v1 [math.mg] 10 Apr 2018 Abstract. The classical Dvoretzky Rogers lemma provies a eterministic
More informationThe derivative of a function f(x) is another function, defined in terms of a limiting expression: f(x + δx) f(x)
Y. D. Chong (2016) MH2801: Complex Methos for the Sciences 1. Derivatives The erivative of a function f(x) is another function, efine in terms of a limiting expression: f (x) f (x) lim x δx 0 f(x + δx)
More informationTensors, Fields Pt. 1 and the Lie Bracket Pt. 1
Tensors, Fiels Pt. 1 an the Lie Bracket Pt. 1 PHYS 500 - Southern Illinois University September 8, 2016 PHYS 500 - Southern Illinois University Tensors, Fiels Pt. 1 an the Lie Bracket Pt. 1 September 8,
More informationLinear First-Order Equations
5 Linear First-Orer Equations Linear first-orer ifferential equations make up another important class of ifferential equations that commonly arise in applications an are relatively easy to solve (in theory)
More informationLarge Triangles in the d-dimensional Unit-Cube (Extended Abstract)
Large Triangles in the -Dimensional Unit-Cube Extene Abstract) Hanno Lefmann Fakultät für Informatik, TU Chemnitz, D-0907 Chemnitz, Germany lefmann@informatik.tu-chemnitz.e Abstract. We consier a variant
More informationMonotonicity of facet numbers of random convex hulls
Monotonicity of facet numbers of ranom convex hulls Gilles Bonnet, Julian Grote, Daniel Temesvari, Christoph Thäle, Nicola Turchi an Florian Wespi arxiv:173.31v1 [math.mg] 7 Mar 17 Abstract Let X 1,...,
More informationCHAPTER 1 : DIFFERENTIABLE MANIFOLDS. 1.1 The definition of a differentiable manifold
CHAPTER 1 : DIFFERENTIABLE MANIFOLDS 1.1 The efinition of a ifferentiable manifol Let M be a topological space. This means that we have a family Ω of open sets efine on M. These satisfy (1), M Ω (2) the
More informationOn the number of isolated eigenvalues of a pair of particles in a quantum wire
On the number of isolate eigenvalues of a pair of particles in a quantum wire arxiv:1812.11804v1 [math-ph] 31 Dec 2018 Joachim Kerner 1 Department of Mathematics an Computer Science FernUniversität in
More informationSelf-normalized Martingale Tail Inequality
Online-to-Confience-Set Conversions an Application to Sparse Stochastic Banits A Self-normalize Martingale Tail Inequality The self-normalize martingale tail inequality that we present here is the scalar-value
More informationCONTROL CHARTS FOR VARIABLES
UNIT CONTOL CHATS FO VAIABLES Structure.1 Introuction Objectives. Control Chart Technique.3 Control Charts for Variables.4 Control Chart for Mean(-Chart).5 ange Chart (-Chart).6 Stanar Deviation Chart
More informationThe total derivative. Chapter Lagrangian and Eulerian approaches
Chapter 5 The total erivative 51 Lagrangian an Eulerian approaches The representation of a flui through scalar or vector fiels means that each physical quantity uner consieration is escribe as a function
More informationA note on asymptotic formulae for one-dimensional network flow problems Carlos F. Daganzo and Karen R. Smilowitz
A note on asymptotic formulae for one-imensional network flow problems Carlos F. Daganzo an Karen R. Smilowitz (to appear in Annals of Operations Research) Abstract This note evelops asymptotic formulae
More informationALGEBRAIC AND ANALYTIC PROPERTIES OF ARITHMETIC FUNCTIONS
ALGEBRAIC AND ANALYTIC PROPERTIES OF ARITHMETIC FUNCTIONS MARK SCHACHNER Abstract. When consiere as an algebraic space, the set of arithmetic functions equippe with the operations of pointwise aition an
More informationRamsey numbers of some bipartite graphs versus complete graphs
Ramsey numbers of some bipartite graphs versus complete graphs Tao Jiang, Michael Salerno Miami University, Oxfor, OH 45056, USA Abstract. The Ramsey number r(h, K n ) is the smallest positive integer
More informationResistant Polynomials and Stronger Lower Bounds for Depth-Three Arithmetical Formulas
Resistant Polynomials an Stronger Lower Bouns for Depth-Three Arithmetical Formulas Maurice J. Jansen an Kenneth W.Regan University at Buffalo (SUNY) Abstract. We erive quaratic lower bouns on the -complexity
More informationHomework 3 - Solutions
Homework 3 - Solutions The Transpose an Partial Transpose. 1 Let { 1, 2,, } be an orthonormal basis for C. The transpose map efine with respect to this basis is a superoperator Γ that acts on an operator
More informationSchrödinger s equation.
Physics 342 Lecture 5 Schröinger s Equation Lecture 5 Physics 342 Quantum Mechanics I Wenesay, February 3r, 2010 Toay we iscuss Schröinger s equation an show that it supports the basic interpretation of
More information