Randomized Algorithms
|
|
- Eric Blake
- 5 years ago
- Views:
Transcription
1 Randomized Algorithms 南京大学 尹一通
2 Martingales Definition: A sequence of random variables X 0, X 1,... is a martingale if for all i > 0, E[X i X 0,...,X i1 ] = X i1 x 0, x 1,...,x i1, E[X i X 0 = x 0, X 1 = x 1,...,X i1 = x i1 ] = x i1
3 Generalization Definition: Y 0,Y 1,... is a martingale with respect to X 0, X 1,... if, for all i 0, Y i is a function of X 0, X 1,...,X i ; E[Y i+1 X 0,...,X i ] = Y i.
4 Azuma s Inequality (general version): Let Y 0,Y 1,... be a martingale with respect to X 0, X 1,... such that, for all k 1, Then Y k Y k1 c k, Pr[ Y n Y 0 t] 2exp t 2 2 n k=1 c2 k.
5 Doob Sequence Definition (Doob sequence): The Doob sequence of a function f with respect to a sequence X 1,...,X n is Y i = E[f (X 1,...,X n ) X 1,...,X i ] Y 0 = E[f (X 1,...,X n )] Y n = f (X 1,...,X n )
6 Doob Martingale f
7 Doob sequence: Y i = E[f (X 1,...,X n ) X 1,...,X i ] Doob sequence is a martingale: E[Y i X 1,...,X i 1 ]=Y i 1 Proof: E[Y i X 1,...,X i 1 ] = E[E[f(X 1,...,X n ) X 1,...,X i ] X 1,...,X i 1 ] = E[f(X 1,...,X n ) X 1,...,X i 1 ] = Y i 1
8 The Power of Doob + Azuma For a function of (dependent) random variables: f (X 1,...,X n ) Doob martingale: Y i = E[f (X 1,...,X n ) X 1,...,X i ] Y 0 = E[f (X 1,...,X n )] Y n = f (X 1,...,X n ) If the differences Y i Y i1 are bounded, Azuma Y n Y 0 is small whp. f (X 1,...,X n ) is tightly concentrated to its mean
9 The method of bounded differences: Let X = (X 1,...,X n ) and let f be a function of X 0, X 1,...,X n satisfying that, for all 1 i n, E[f (X ) X 1,...,X i ] E[f (X ) X 1,...,X i1 ] c i, Then Pr f (X ) E[f (X )] t 2exp t 2 2 n i=1 c2 i.
10 The method of bounded differences: Let X = (X 1,...,X n ) and let f be a function of X 0, X 1,...,X n satisfying that, for all 1 i n, E[f (X ) X 1,...,X i ] E[f (X ) X 1,...,X i1 ] c i, Then (Azuma) Y i Y i 1 Pr f (X ) E[f (X )] t 2exp Y n Y 0 t 2 2 n i=1 c2 i. Doob martingale: Y i = E[f (X ) X 1,...,X i ]
11 G(n, p) chromatic number: (G) the smallest number of colors to properly color G X i : subgraph of G induced by the first i vertices Y i = E[ (G) X 1,...,X i ] Doob martingale: Y 0,Y 1,...,Y n Y 0 = E[ (G)] Y n = (G)
12 chromatic number: (G) X i : subgraph of G induced by the first i vertices Y i = E[ (G) X 1,...,X i ] Doob martingale: Y 0,Y 1,...,Y n Y 0 = E[ (G)] Y n = (G) Observation: a vertex can always be given a new color. Y i Y i 1 1 Pr[ (G) E[ (G)] t n] = Pr[ Y n Y 0 t n] 2e t2 /2
13 Tight Concentration of Chromatic Number Theorem [Shamir & Spencer (1987)]: Let G G(n, p). Then Pr (G) E (G) t n 2e t 2 /2.
14 Hoeffding s Inequality Hoeffding s Inequality: Let X = n i=1 X i, where X 1,...,X n are independent random variables with a i X i b i. Then Pr[ X E[X ] t] 2exp 2 n i=1 (b i a i ) 2 t 2
15 Hoeffding s Inequality: Let X = n i=1 X i, where X 1,...,X n are independent random variables with Then a i X i b i. Pr[ X E[X ] t] 2exp 2 n i=1 (b i a i ) 2 t 2 Proof: X is a function (sum) of X 1,...,X n. Doob martingale: Y i = E[X X 1,...,X i ] Y i Y i1 = X i E[X i ] b i a i Azuma Done!
16 The method of bounded differences: Let X = (X 1,...,X n ) and let f be a function of X 0, X 1,...,X n satisfying that, for all 1 i n, E[f (X ) X 1,...,X i ] E[f (X ) X 1,...,X i1 ] c i, Then hard to check! Pr f (X ) E[f (X )] t 2exp t 2 2 n i=1 c2 i.
17 E[f (X ) X 1,...,X i ] E[f (X ) X 1,...,X i1 ] c i, Lipschitz Condition: f (x 1,...,x n ) satisfies the Lipschitz condition with constants c i,1 i n, if f (x 1,...,x i 1, x i, x i+1,...,x n ) f (x 1,...,x i 1, y i, x i+1,...,x n ) c i.
18 Average-case: E[f (X ) X 1,...,X i ] E[f (X ) X 1,...,X i1 ] c i, Worst-case: Lipschitz Condition: f (x 1,...,x n ) satisfies the Lipschitz condition with constants c i,1 i n, if f (x 1,...,x i 1, x i, x i+1,...,x n ) f (x 1,...,x i 1, y i, x i+1,...,x n ) c i.
19 The method of bounded differences: Let X = (X 1,...,X n ) be n independent random variables and let f be a function satisfying the Lipschitz condition with constants c i,1 i n. Then Pr f (X ) E[f (X )] t 2exp t 2 2 n i=1 c2 i.
20 The method of bounded differences: Let X = (X 1,...,X n ) be n independent random variables and let f be a function satisfying the Lipschitz condition with constants c i,1 i n. Then Pr f (X ) E[f (X )] t 2exp t 2 2 n i=1 c2 i. Proof: Lipschitz condition + independence bounded averaged differences
21 Applications of the Method A function f of independent random variable: X 1, X 2,...,X n Lipschitz condition of f: changes of any variable makes little change to the value of f. f (X 1, X 2,...,X n ) is tightly concentrated to its mean.
22 Occupancy Problem m-balls-into-n-bins: number of empty bins? 1 bin i is empty X i = 0 otherwise # empty bins: X = n X i i=1 E[X ] = n i=1 E[X i ] = n 1 1 n m Xi are deviation: Pr[ X E[X ] t]? dependent
23 Occupancy Problem m-balls-into-n-bins: # empty bins: X number of empty bins? Y j : the bin of ball j deviation: (Independent!) Pr[ X E[X ] t]? X = f (Y 1,...,Y m ) = [n] {Y 1,...,Y m } Lipschitz: changing any Y j can change X for at most 1 Pr[ X E[X ] t m] 2e t 2 /2
24 Pattern Matching a random string of length n, a pattern of length k, # of matched substrings? alphabet Σ =m a fixed pattern: k uniform & independent: X 1,...,X n Y : #substrings in (X 1,...,X n ) E[Y ] = (n k + 1) 1 m k Deviation?
25 Pattern Matching a random string of length n, a pattern of length k, # of matched substrings? alphabet Σ =m a fixed pattern: k uniform & independent: X 1,...,X n Y = f (X 1,...,X n ) changing any X i changes f for at most k Pr Y E[Y ] tk n 2e t 2 /2
26 Metric Embedding (X, d X ) (Y,d Y ) low-distortion: For a small 1 x 1,x 2 X, 1 dx (x 1,x 2 ) d Y ((x 1 ), (x 2 )) d X (x 1,x 2 )
27 Why Embedding? Some problems are easier to solve in simple metrics: TSP in trees. Curse of dimensionality proximity search; learning; due to volume explosion graph metric high dimension tree metric low dimension
28 Dimension Reduction In Euclidian space, it is always possible to embed a set of n points in arbitrary dimension to O(log n) dimension with constant distortion. Johnson-Lindenstrauss Theorem: For any 0 < < 1, for any set V of n points in R d, there is a map : R d R k with k = O(ln n), such that u, v V, (1 ) u v 2 (u) (v) 2 (1 + ) u v 2
29 Johnson-Lindenstrauss Theorem Johnson-Lindenstrauss Theorem: For any 0 < < 1, for any set V of n points in R d, there is a map : R d R k with k = O(ln n), such that u, v V, (1 ) u v 2 (u) (v) 2 (1 + ) u v 2 (v) =Av. A is a random projection matrix.
30 Random Projection Random k d matrix A: Projection onto a uniform random subspace. (Johnson-Lindenstrauss) (Dasgupta-Gupta) i.i.d. Gaussian entries. (Indyk-Motiwani) i.i.d. -1/+1 entries. (Achlioptas) rows: A 1,A 2,...,A k random orthogonal unit vectors R d
31 Johnson-Lindenstrauss Theorem Let V be a set of n points in R d. For some k = O(ln n). Let A be a random k d matrix that projects onto a uniform random k-dimensional subspace. W.h.p., u, v V, u v 2 d 2 k Au (for the case ε=1/2) 2 d k Av 3 u v 2 2 the projection
32 Dasgupta-Gupta Proof Outline: 1. Reduce to the distortion to unit vectors. 2. Reduce to a fixed projection of random unit vector. 3. Bound the norm of the first k entries of uniform random unit vector. V : n point in R d. A: projection onto a uniform random k-subspace k = O(ln n) W.h.p., u, v V, d k Au d k Au d k Av d k Av u u 2 v 2 v 2 2
33 A: projection onto a uniform random k-subspace O(n 2 ) pairs W.h.p., u, v V, d k Au d k Av 2 3 u 2 v 2 d k Au d k Av 2 u 2 v 2 A (u v) u v 2 3k 2d A (u v) u v 2 k 2d treat (u v) as a vector, u v u v is a unit vector New goal: unit vector u, with probability o( 1 n 2 ) Au 2 > 3k 2d or Au 2 < k 2d
34 A: projection onto a uniform random k-subspace unit vector u, with probability o( 1 n 2 ) Au 2 > 3k 2d or Au 2 < k 2d probabilistically equivalent random unit vector random subspace fixed unit vector fixed subspace Au
35 A: projection onto a uniform random k-subspace unit vector u, with probability o( 1 n 2 ) Au 2 > 3k 2d or Au 2 < k 2d uniform random unit vector: random unit vector Y =(Y 1,...,Y k,y k+1,...,y d ) Z =(Y 1,...,Y k ) Z 2 > 3k 2d or Z 2 < k 2d fixed subspace
36 uniform random unit vector: Y =(Y 1,...,Y k,y k+1,...,y d ) Let Z =(Y 1,...,Y k ) with probability o( 1 n 2 ) Z 2 > 3k 2d or Z 2 < k 2d Looks clean, however... Z 2 = Y1 2 + Y Yk 2 Y i s are dependent for uniform unit vector Y.
37 uniform random unit vector: Y =(Y 1,...,Y k,y k+1,...,y d ) Let Z =(Y 1,...,Y k ) Generating uniform unit vector: X 1,...,X d are i.i.d. normal distributions N(0, 1). X =(X 1,...,X d ), Y = 1 X X Y is a uniform random unit vector. The density: Pr[(x 1,...,x d )] = d i=1 1 2 e x 2 i /2 = (2 ) d/2 e x2 /2 Spherically symmetric!
38 X 1,...,X d are i.i.d. normal distributions N(0, 1). X =(X 1,...,X d ), Z = 1 X (X 1,...,X k ) Z 2 = X X 2 k X X2 d Z 2 < k 2d Z 2 > 3k 2d (k 2d) (3k 2d) k i=1 k X 2 i + k X 2 i +3k d i=k+1 d i=1 i=k+1 X 2 i > 0 X 2 i < 0 Sum of independent random variables! Chernoff?
39 X 1,...,X d are i.i.d. normal distributions N(0, 1). Upper bound: (for > 0) (Markov) (independence) = Pr E = Pr k i=1 exp exp E (k 2d) e (k k i=1 (k 2d) (k 2d) 2d)X 2 i X 2 i + k k i=1 k i=1 d i=k+1 X 2 i + k X 2 i + k E d i=k+1 e kx2 i d X 2 i i=k+1 d X 2 i i=k+1 X 2 i > 0 > 1 E e sx2 i = 1 1 2s.
40 X 1,...,X d are i.i.d. normal distributions N(0, 1). Upper bound: (for > 0) (Markov) (independence) = Pr E = Pr k i=1 exp exp E (k 2d) e (k k i=1 (k 2d) (k 2d) 2d)X 2 i X 2 i + k k i=1 k i=1 d i=k+1 X 2 i + k X 2 i + k E d i=k+1 e kx2 i d X 2 i i=k+1 d X 2 i i=k+1 X 2 i > 0 > 1 E e sx2 i = 1 1 2s.
41 X 1,...,X d are i.i.d. normal distributions N(0, 1). Upper bound: (for > 0) (Markov) (independence) = Pr E = Pr k i=1 exp exp E (k 2d) e (k k i=1 (k 2d) (k 2d) 2d)X 2 i X 2 i + k k i=1 k i=1 d i=k+1 X 2 i + k X 2 i + k E d i=k+1 e kx2 i d X 2 i i=k+1 d X 2 i i=k+1 X 2 i > 0 > 1 E e sx2 i = 1 1 2s. = (1 2 (k 2d)) k d k 2 (1 2 k) 2
42 X 1,...,X d are i.i.d. normal distributions N(0, 1). Pr (k 2d) k i=1 X 2 i + k d i=k+1 X 2 i > 0 (1 2 (k 2d)) k d k 2 (1 2 k) 2 e k/16 minimized when = 1 2d k k Pr (3k 2d) Xi 2 +3k Xi 2 < 0 e k/24 d i=1 i=k+1 Z = 1 X (X 1,...,X k ) Pr Z 2 > 3k 2d for some k = O(ln n) Z 2 < k = o( 1 2d n 3 )
43 for some k = O(ln n) Z: fixed k-subspace of random unit vector. Pr Z 2 distorted from k/d = o( 1 n 3 ) Au: random k-subspace of fixed unit vector. Pr Au 2 distorted from k/d = o( 1 n 3 ) u, v Pr d/ka(u v) 2 distorted from u v 2 = o( 1 n 3 ) O(n 2 ) pairs of (u, v). Union bound Johnson- Lindenstrauss
44 Johnson-Lindenstrauss Theorem Johnson-Lindenstrauss Theorem: For any 0 < < 1, for any set V of n points in R d, there is a map : R d R k with k = O(ln n), such that u, v V, (1 ) u v 2 (u) (v) 2 (1 + ) u v 2 (v) =Av. A is a random projection matrix.
45 Random Projection Random k d matrix A: Projection onto a uniform random subspace. (Johnson-Lindenstrauss) (Dasgupta-Gupta) i.i.d. Gaussian entries. (Indyk-Motiwani) i.i.d. -1/+1 entries. (Achlioptas) rows: A 1,A 2,...,A k random orthogonal unit vectors R d
Outline. Martingales. Piotr Wojciechowski 1. 1 Lane Department of Computer Science and Electrical Engineering West Virginia University.
Outline Piotr 1 1 Lane Department of Computer Science and Electrical Engineering West Virginia University 8 April, 01 Outline Outline 1 Tail Inequalities Outline Outline 1 Tail Inequalities General Outline
More information16 Embeddings of the Euclidean metric
16 Embeddings of the Euclidean metric In today s lecture, we will consider how well we can embed n points in the Euclidean metric (l 2 ) into other l p metrics. More formally, we ask the following question.
More informationLecture 18: March 15
CS71 Randomness & Computation Spring 018 Instructor: Alistair Sinclair Lecture 18: March 15 Disclaimer: These notes have not been subjected to the usual scrutiny accorded to formal publications. They may
More informationSTAT 200C: High-dimensional Statistics
STAT 200C: High-dimensional Statistics Arash A. Amini May 30, 2018 1 / 59 Classical case: n d. Asymptotic assumption: d is fixed and n. Basic tools: LLN and CLT. High-dimensional setting: n d, e.g. n/d
More informationRandomized Algorithms
Randomized Algorithms Saniv Kumar, Google Research, NY EECS-6898, Columbia University - Fall, 010 Saniv Kumar 9/13/010 EECS6898 Large Scale Machine Learning 1 Curse of Dimensionality Gaussian Mixture Models
More informationAn Algorithmist s Toolkit Nov. 10, Lecture 17
8.409 An Algorithmist s Toolkit Nov. 0, 009 Lecturer: Jonathan Kelner Lecture 7 Johnson-Lindenstrauss Theorem. Recap We first recap a theorem (isoperimetric inequality) and a lemma (concentration) from
More informationLecture 5: Probabilistic tools and Applications II
T-79.7003: Graphs and Networks Fall 2013 Lecture 5: Probabilistic tools and Applications II Lecturer: Charalampos E. Tsourakakis Oct. 11, 2013 5.1 Overview In the first part of today s lecture we will
More information18.409: Topics in TCS: Embeddings of Finite Metric Spaces. Lecture 6
Massachusetts Institute of Technology Lecturer: Michel X. Goemans 8.409: Topics in TCS: Embeddings of Finite Metric Spaces September 7, 006 Scribe: Benjamin Rossman Lecture 6 Today we loo at dimension
More informationHamming Cube and Other Stuff
Hamming Cube and Other Stuff Sabrina Sixta Tuesday, May 27, 2014 Sabrina Sixta () Hamming Cube and Other Stuff Tuesday, May 27, 2014 1 / 12 Table of Contents Outline 1 Definition of a Hamming cube 2 Properties
More informationConcentration function and other stuff
Concentration function and other stuff Sabrina Sixta Tuesday, June 16, 2014 Sabrina Sixta () Concentration function and other stuff Tuesday, June 16, 2014 1 / 13 Table of Contents Outline 1 Chernoff Bound
More information1 Dimension Reduction in Euclidean Space
CSIS0351/8601: Randomized Algorithms Lecture 6: Johnson-Lindenstrauss Lemma: Dimension Reduction Lecturer: Hubert Chan Date: 10 Oct 011 hese lecture notes are supplementary materials for the lectures.
More informationVery Sparse Random Projections
Very Sparse Random Projections Ping Li, Trevor Hastie and Kenneth Church [KDD 06] Presented by: Aditya Menon UCSD March 4, 2009 Presented by: Aditya Menon (UCSD) Very Sparse Random Projections March 4,
More informationEmbeddings of finite metric spaces in Euclidean space: a probabilistic view
Embeddings of finite metric spaces in Euclidean space: a probabilistic view Yuval Peres May 11, 2006 Talk based on work joint with: Assaf Naor, Oded Schramm and Scott Sheffield Definition: An invertible
More informationSTAT 200C: High-dimensional Statistics
STAT 200C: High-dimensional Statistics Arash A. Amini April 27, 2018 1 / 80 Classical case: n d. Asymptotic assumption: d is fixed and n. Basic tools: LLN and CLT. High-dimensional setting: n d, e.g. n/d
More informationAdvances in Manifold Learning Presented by: Naku Nak l Verm r a June 10, 2008
Advances in Manifold Learning Presented by: Nakul Verma June 10, 008 Outline Motivation Manifolds Manifold Learning Random projection of manifolds for dimension reduction Introduction to random projections
More informationLectures 6: Degree Distributions and Concentration Inequalities
University of Washington Lecturer: Abraham Flaxman/Vahab S Mirrokni CSE599m: Algorithms and Economics of Networks April 13 th, 007 Scribe: Ben Birnbaum Lectures 6: Degree Distributions and Concentration
More informationEssentials on the Analysis of Randomized Algorithms
Essentials on the Analysis of Randomized Algorithms Dimitris Diochnos Feb 0, 2009 Abstract These notes were written with Monte Carlo algorithms primarily in mind. Topics covered are basic (discrete) random
More informationMultivariate Statistics Random Projections and Johnson-Lindenstrauss Lemma
Multivariate Statistics Random Projections and Johnson-Lindenstrauss Lemma Suppose again we have n sample points x,..., x n R p. The data-point x i R p can be thought of as the i-th row X i of an n p-dimensional
More informationJOHNSON-LINDENSTRAUSS TRANSFORMATION AND RANDOM PROJECTION
JOHNSON-LINDENSTRAUSS TRANSFORMATION AND RANDOM PROJECTION LONG CHEN ABSTRACT. We give a brief survey of Johnson-Lindenstrauss lemma. CONTENTS 1. Introduction 1 2. JL Transform 4 2.1. An Elementary Proof
More informationFinite Metric Spaces & Their Embeddings: Introduction and Basic Tools
Finite Metric Spaces & Their Embeddings: Introduction and Basic Tools Manor Mendel, CMI, Caltech 1 Finite Metric Spaces Definition of (semi) metric. (M, ρ): M a (finite) set of points. ρ a distance function
More informationNon-Asymptotic Theory of Random Matrices Lecture 4: Dimension Reduction Date: January 16, 2007
Non-Asymptotic Theory of Random Matrices Lecture 4: Dimension Reduction Date: January 16, 2007 Lecturer: Roman Vershynin Scribe: Matthew Herman 1 Introduction Consider the set X = {n points in R N } where
More informationAdvanced Algorithms 南京大学 尹一通
Advanced Algorithms 南京大学 尹一通 Constraint Satisfaction Problem variables: (CSP) X = {x 1, x2,..., xn} each variable ranges over a finite domain Ω an assignment σ ΩX assigns each variable a value in Ω constraints:
More information11.1 Set Cover ILP formulation of set cover Deterministic rounding
CS787: Advanced Algorithms Lecture 11: Randomized Rounding, Concentration Bounds In this lecture we will see some more examples of approximation algorithms based on LP relaxations. This time we will use
More informationFast Dimension Reduction
Fast Dimension Reduction Nir Ailon 1 Edo Liberty 2 1 Google Research 2 Yale University Introduction Lemma (Johnson, Lindenstrauss (1984)) A random projection Ψ preserves all ( n 2) distances up to distortion
More informationSome Useful Background for Talk on the Fast Johnson-Lindenstrauss Transform
Some Useful Background for Talk on the Fast Johnson-Lindenstrauss Transform Nir Ailon May 22, 2007 This writeup includes very basic background material for the talk on the Fast Johnson Lindenstrauss Transform
More information15-854: Approximation Algorithms Lecturer: Anupam Gupta Topic: Approximating Metrics by Tree Metrics Date: 10/19/2005 Scribe: Roy Liu
15-854: Approximation Algorithms Lecturer: Anupam Gupta Topic: Approximating Metrics by Tree Metrics Date: 10/19/2005 Scribe: Roy Liu 121 Introduction In this lecture, we show how to embed metric weighted
More informationECE534, Spring 2018: Solutions for Problem Set #4 Due Friday April 6, 2018
ECE534, Spring 2018: s for Problem Set #4 Due Friday April 6, 2018 1. MMSE Estimation, Data Processing and Innovations The random variables X, Y, Z on a common probability space (Ω, F, P ) are said to
More informationSparse Johnson-Lindenstrauss Transforms
Sparse Johnson-Lindenstrauss Transforms Jelani Nelson MIT May 24, 211 joint work with Daniel Kane (Harvard) Metric Johnson-Lindenstrauss lemma Metric JL (MJL) Lemma, 1984 Every set of n points in Euclidean
More informationAccelerated Dense Random Projections
1 Advisor: Steven Zucker 1 Yale University, Department of Computer Science. Dimensionality reduction (1 ε) xi x j 2 Ψ(xi ) Ψ(x j ) 2 (1 + ε) xi x j 2 ( n 2) distances are ε preserved Target dimension k
More informationThe Johnson-Lindenstrauss Lemma
The Johnson-Lindenstrauss Lemma Kevin (Min Seong) Park MAT477 Introduction The Johnson-Lindenstrauss Lemma was first introduced in the paper Extensions of Lipschitz mappings into a Hilbert Space by William
More informationSparser Johnson-Lindenstrauss Transforms
Sparser Johnson-Lindenstrauss Transforms Jelani Nelson Princeton February 16, 212 joint work with Daniel Kane (Stanford) Random Projections x R d, d huge store y = Sx, where S is a k d matrix (compression)
More informationProbabilistic Combinatorics. Jeong Han Kim
Probabilistic Combinatorics Jeong Han Kim 1 Tentative Plans 1. Basics for Probability theory Coin Tossing, Expectation, Linearity of Expectation, Probability vs. Expectation, Bool s Inequality 2. First
More informationUniform concentration inequalities, martingales, Rademacher complexity and symmetrization
Uniform concentration inequalities, martingales, Rademacher complexity and symmetrization John Duchi Outline I Motivation 1 Uniform laws of large numbers 2 Loss minimization and data dependence II Uniform
More informationConcentration inequalities and the entropy method
Concentration inequalities and the entropy method Gábor Lugosi ICREA and Pompeu Fabra University Barcelona what is concentration? We are interested in bounding random fluctuations of functions of many
More informationLecture 4: Inequalities and Asymptotic Estimates
CSE 713: Random Graphs and Applications SUNY at Buffalo, Fall 003 Lecturer: Hung Q. Ngo Scribe: Hung Q. Ngo Lecture 4: Inequalities and Asymptotic Estimates We draw materials from [, 5, 8 10, 17, 18].
More informationConcentration inequalities and tail bounds
Concentration inequalities and tail bounds John Duchi Outline I Basics and motivation 1 Law of large numbers 2 Markov inequality 3 Cherno bounds II Sub-Gaussian random variables 1 Definitions 2 Examples
More information25 Minimum bandwidth: Approximation via volume respecting embeddings
25 Minimum bandwidth: Approximation via volume respecting embeddings We continue the study of Volume respecting embeddings. In the last lecture, we motivated the use of volume respecting embeddings by
More informationDimensionality reduction: Johnson-Lindenstrauss lemma for structured random matrices
Dimensionality reduction: Johnson-Lindenstrauss lemma for structured random matrices Jan Vybíral Austrian Academy of Sciences RICAM, Linz, Austria January 2011 MPI Leipzig, Germany joint work with Aicke
More informationLecture 5: January 30
CS71 Randomness & Computation Spring 018 Instructor: Alistair Sinclair Lecture 5: January 30 Disclaimer: These notes have not been subjected to the usual scrutiny accorded to formal publications. They
More informationRandomized algorithm
Tutorial 4 Joyce 2009-11-24 Outline Solution to Midterm Question 1 Question 2 Question 1 Question 2 Question 3 Question 4 Question 5 Solution to Midterm Solution to Midterm Solution to Midterm Question
More informationMethods for sparse analysis of high-dimensional data, II
Methods for sparse analysis of high-dimensional data, II Rachel Ward May 23, 2011 High dimensional data with low-dimensional structure 300 by 300 pixel images = 90, 000 dimensions 2 / 47 High dimensional
More informationMarch 1, Florida State University. Concentration Inequalities: Martingale. Approach and Entropy Method. Lizhe Sun and Boning Yang.
Florida State University March 1, 2018 Framework 1. (Lizhe) Basic inequalities Chernoff bounding Review for STA 6448 2. (Lizhe) Discrete-time martingales inequalities via martingale approach 3. (Boning)
More informationLecture Introduction. 2 Examples of Measure Concentration. 3 The Johnson-Lindenstrauss Lemma. CS-621 Theory Gems November 28, 2012
CS-6 Theory Gems November 8, 0 Lecture Lecturer: Alesaner Mąry Scribes: Alhussein Fawzi, Dorina Thanou Introuction Toay, we will briefly iscuss an important technique in probability theory measure concentration
More informationDissertation Defense
Clustering Algorithms for Random and Pseudo-random Structures Dissertation Defense Pradipta Mitra 1 1 Department of Computer Science Yale University April 23, 2008 Mitra (Yale University) Dissertation
More informationSize and degree anti-ramsey numbers
Size and degree anti-ramsey numbers Noga Alon Abstract A copy of a graph H in an edge colored graph G is called rainbow if all edges of H have distinct colors. The size anti-ramsey number of H, denoted
More informationAlgorithms, Geometry and Learning. Reading group Paris Syminelakis
Algorithms, Geometry and Learning Reading group Paris Syminelakis October 11, 2016 2 Contents 1 Local Dimensionality Reduction 5 1 Introduction.................................... 5 2 Definitions and Results..............................
More informationTail Inequalities. The Chernoff bound works for random variables that are a sum of indicator variables with the same distribution (Bernoulli trials).
Tail Inequalities William Hunt Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV William.Hunt@mail.wvu.edu Introduction In this chapter, we are interested
More informationBasic Properties of Metric and Normed Spaces
Basic Properties of Metric and Normed Spaces Computational and Metric Geometry Instructor: Yury Makarychev The second part of this course is about metric geometry. We will study metric spaces, low distortion
More informationGeneralization Bounds in Machine Learning. Presented by: Afshin Rostamizadeh
Generalization Bounds in Machine Learning Presented by: Afshin Rostamizadeh Outline Introduction to generalization bounds. Examples: VC-bounds Covering Number bounds Rademacher bounds Stability bounds
More informationLecture 13 October 6, Covering Numbers and Maurey s Empirical Method
CS 395T: Sublinear Algorithms Fall 2016 Prof. Eric Price Lecture 13 October 6, 2016 Scribe: Kiyeon Jeon and Loc Hoang 1 Overview In the last lecture we covered the lower bound for p th moment (p > 2) and
More informationNotes taken by Costis Georgiou revised by Hamed Hatami
CSC414 - Metric Embeddings Lecture 6: Reductions that preserve volumes and distance to affine spaces & Lower bound techniques for distortion when embedding into l Notes taken by Costis Georgiou revised
More information6.1 Occupancy Problem
15-859(M): Randomized Algorithms Lecturer: Anupam Gupta Topic: Occupancy Problems and Hashing Date: Sep 9 Scribe: Runting Shi 6.1 Occupancy Problem Bins and Balls Throw n balls into n bins at random. 1.
More informationRandom Projections. Lopez Paz & Duvenaud. November 7, 2013
Random Projections Lopez Paz & Duvenaud November 7, 2013 Random Outline The Johnson-Lindenstrauss Lemma (1984) Random Kitchen Sinks (Rahimi and Recht, NIPS 2008) Fastfood (Le et al., ICML 2013) Why random
More informationMatrix Concentration. Nick Harvey University of British Columbia
Matrix Concentration Nick Harvey University of British Columbia The Problem Given any random nxn, symmetric matrices Y 1,,Y k. Show that i Y i is probably close to E[ i Y i ]. Why? A matrix generalization
More informationImproved Bounds on the Dot Product under Random Projection and Random Sign Projection
Improved Bounds on the Dot Product under Random Projection and Random Sign Projection Ata Kabán School of Computer Science The University of Birmingham Birmingham B15 2TT, UK http://www.cs.bham.ac.uk/
More informationLecture 6 Proof for JL Lemma and Linear Dimensionality Reduction
COMS 4995: Unsupervised Learning (Summer 18) June 7, 018 Lecture 6 Proof for JL Lemma and Linear imensionality Reduction Instructor: Nakul Verma Scribes: Ziyuan Zhong, Kirsten Blancato This lecture gives
More informationThe first bound is the strongest, the other two bounds are often easier to state and compute. Proof: Applying Markov's inequality, for any >0 we have
The first bound is the strongest, the other two bounds are often easier to state and compute Proof: Applying Markov's inequality, for any >0 we have Pr (1 + ) = Pr For any >0, we can set = ln 1+ (4.4.1):
More informationCS5314 Randomized Algorithms. Lecture 18: Probabilistic Method (De-randomization, Sample-and-Modify)
CS5314 Randomized Algorithms Lecture 18: Probabilistic Method (De-randomization, Sample-and-Modify) 1 Introduce two topics: De-randomize by conditional expectation provides a deterministic way to construct
More informationContribution from: Springer Verlag Berlin Heidelberg 2005 ISBN
Contribution from: Mathematical Physics Studies Vol. 7 Perspectives in Analysis Essays in Honor of Lennart Carleson s 75th Birthday Michael Benedicks, Peter W. Jones, Stanislav Smirnov (Eds.) Springer
More information17 Random Projections and Orthogonal Matching Pursuit
17 Random Projections and Orthogonal Matching Pursuit Again we will consider high-dimensional data P. Now we will consider the uses and effects of randomness. We will use it to simplify P (put it in a
More informationVertex colorings of graphs without short odd cycles
Vertex colorings of graphs without short odd cycles Andrzej Dudek and Reshma Ramadurai Department of Mathematical Sciences Carnegie Mellon University Pittsburgh, PA 1513, USA {adudek,rramadur}@andrew.cmu.edu
More informationA Simple Algorithm for Clustering Mixtures of Discrete Distributions
1 A Simple Algorithm for Clustering Mixtures of Discrete Distributions Pradipta Mitra In this paper, we propose a simple, rotationally invariant algorithm for clustering mixture of distributions, including
More informationInduced subgraphs with many repeated degrees
Induced subgraphs with many repeated degrees Yair Caro Raphael Yuster arxiv:1811.071v1 [math.co] 17 Nov 018 Abstract Erdős, Fajtlowicz and Staton asked for the least integer f(k such that every graph with
More informationLecture 1 Measure concentration
CSE 29: Learning Theory Fall 2006 Lecture Measure concentration Lecturer: Sanjoy Dasgupta Scribe: Nakul Verma, Aaron Arvey, and Paul Ruvolo. Concentration of measure: examples We start with some examples
More informationMidterm. CS265/CME309, Fall Instructor: Gregory Valiant. Name: SUID Number:
CS265/CME309, Fall 2016. Instructor: Gregory Valiant Name: Midterm SUID Number: [This is a closed-notes/closed-computer exam, though you may refer to 1 page (or 2 sides) of 8.5 x 11 notes that you have
More informationLecture 4: Applications: random trees, determinantal measures and sampling
Lecture 4: Applications: random trees, determinantal measures and sampling Robin Pemantle University of Pennsylvania pemantle@math.upenn.edu Minerva Lectures at Columbia University 09 November, 2016 Sampling
More informationMartingale techniques
Chapter 3 Martingale techniques Martingales are a central tool in probability theory. In this chapter we illustrate their use, as well as some related concepts, on a number of applications in discrete
More informationBalanced Allocation Through Random Walk
Balanced Allocation Through Random Walk Alan Frieze Samantha Petti November 25, 2017 Abstract We consider the allocation problem in which m (1 ε)dn items are to be allocated to n bins with capacity d.
More informationSelected Exercises on Expectations and Some Probability Inequalities
Selected Exercises on Expectations and Some Probability Inequalities # If E(X 2 ) = and E X a > 0, then P( X λa) ( λ) 2 a 2 for 0 < λ
More informationMAT 585: Johnson-Lindenstrauss, Group testing, and Compressed Sensing
MAT 585: Johnson-Lindenstrauss, Group testing, and Compressed Sensing Afonso S. Bandeira April 9, 2015 1 The Johnson-Lindenstrauss Lemma Suppose one has n points, X = {x 1,..., x n }, in R d with d very
More informationSketching as a Tool for Numerical Linear Algebra All Lectures. David Woodruff IBM Almaden
Sketching as a Tool for Numerical Linear Algebra All Lectures David Woodruff IBM Almaden Massive data sets Examples Internet traffic logs Financial data etc. Algorithms Want nearly linear time or less
More informationGraph-Constrained Group Testing
Mahdi Cheraghchi et al. presented by Ying Xuan May 20, 2010 Table of contents 1 Motivation and Problem Definitions 2 3 Correctness Proof 4 Applications call for connected pools detection of congested links
More informationRandom Methods for Linear Algebra
Gittens gittens@acm.caltech.edu Applied and Computational Mathematics California Institue of Technology October 2, 2009 Outline The Johnson-Lindenstrauss Transform 1 The Johnson-Lindenstrauss Transform
More information< k 2n. 2 1 (n 2). + (1 p) s) N (n < 1
List of Problems jacques@ucsd.edu Those question with a star next to them are considered slightly more challenging. Problems 9, 11, and 19 from the book The probabilistic method, by Alon and Spencer. Question
More informationConcentration of Measures by Bounded Couplings
Concentration of Measures by Bounded Couplings Subhankar Ghosh, Larry Goldstein and Ümit Işlak University of Southern California [arxiv:0906.3886] [arxiv:1304.5001] May 2013 Concentration of Measure Distributional
More informationAssignment 4: Solutions
Math 340: Discrete Structures II Assignment 4: Solutions. Random Walks. Consider a random walk on an connected, non-bipartite, undirected graph G. Show that, in the long run, the walk will traverse each
More informationThe Johnson-Lindenstrauss Lemma for Linear and Integer Feasibility
The Johnson-Lindenstrauss Lemma for Linear and Integer Feasibility Leo Liberti, Vu Khac Ky, Pierre-Louis Poirion CNRS LIX Ecole Polytechnique, France Rutgers University, 151118 The gist I want to solve
More informationRandomized Algorithms Week 2: Tail Inequalities
Randomized Algorithms Week 2: Tail Inequalities Rao Kosaraju In this section, we study three ways to estimate the tail probabilities of random variables. Please note that the more information we know about
More informationProbabilistic Method. Benny Sudakov. Princeton University
Probabilistic Method Benny Sudakov Princeton University Rough outline The basic Probabilistic method can be described as follows: In order to prove the existence of a combinatorial structure with certain
More informationFaster Johnson-Lindenstrauss style reductions
Faster Johnson-Lindenstrauss style reductions Aditya Menon August 23, 2007 Outline 1 Introduction Dimensionality reduction The Johnson-Lindenstrauss Lemma Speeding up computation 2 The Fast Johnson-Lindenstrauss
More informationEntropy and Ergodic Theory Lecture 15: A first look at concentration
Entropy and Ergodic Theory Lecture 15: A first look at concentration 1 Introduction to concentration Let X 1, X 2,... be i.i.d. R-valued RVs with common distribution µ, and suppose for simplicity that
More informationA Note on Discrete Gaussian Combinations of Lattice Vectors
CHICAGO JOURNAL OF THEORETICAL COMPUTER SCIENCE 2016, Article 07, pages 1 11 http://cjtcs.cs.uchicago.edu/ A Note on Discrete Gaussian Combinations of Lattice Vectors Divesh Aggarwal Oded Regev * Received
More informationSliding Windows with Limited Storage
Electronic Colloquium on Computational Complexity, Report No. 178 (2012) Sliding Windows with Limited Storage Paul Beame Computer Science and Engineering University of Washington Seattle, WA 98195-2350
More informationAssignment 2 : Probabilistic Methods
Assignment 2 : Probabilistic Methods jverstra@math Question 1. Let d N, c R +, and let G n be an n-vertex d-regular graph. Suppose that vertices of G n are selected independently with probability p, where
More informationRandom Feature Maps for Dot Product Kernels Supplementary Material
Random Feature Maps for Dot Product Kernels Supplementary Material Purushottam Kar and Harish Karnick Indian Institute of Technology Kanpur, INDIA {purushot,hk}@cse.iitk.ac.in Abstract This document contains
More informationPreliminaries and Complexity Theory
Preliminaries and Complexity Theory Oleksandr Romanko CAS 746 - Advanced Topics in Combinatorial Optimization McMaster University, January 16, 2006 Introduction Book structure: 2 Part I Linear Algebra
More informationConcentration of Measures by Bounded Size Bias Couplings
Concentration of Measures by Bounded Size Bias Couplings Subhankar Ghosh, Larry Goldstein University of Southern California [arxiv:0906.3886] January 10 th, 2013 Concentration of Measure Distributional
More informationA Unified Theorem on SDP Rank Reduction. yyye
SDP Rank Reduction Yinyu Ye, EURO XXII 1 A Unified Theorem on SDP Rank Reduction Yinyu Ye Department of Management Science and Engineering and Institute of Computational and Mathematical Engineering Stanford
More informationNon-Uniform Graph Partitioning
Robert Seffi Roy Kunal Krauthgamer Naor Schwartz Talwar Weizmann Institute Technion Microsoft Research Microsoft Research Problem Definition Introduction Definitions Related Work Our Result Input: G =
More informationAdvanced Algorithms 南京大学 尹一通
Advanced Algorithms 南京大学 尹一通 LP-based Algorithms LP rounding: Relax the integer program to LP; round the optimal LP solution to a nearby feasible integral solution. The primal-dual schema: Find a pair
More informationRandom projections. 1 Introduction. 2 Dimensionality reduction. Lecture notes 5 February 29, 2016
Lecture notes 5 February 9, 016 1 Introduction Random projections Random projections are a useful tool in the analysis and processing of high-dimensional data. We will analyze two applications that use
More informationList coloring hypergraphs
List coloring hypergraphs Penny Haxell Jacques Verstraete Department of Combinatorics and Optimization University of Waterloo Waterloo, Ontario, Canada pehaxell@uwaterloo.ca Department of Mathematics University
More informationOptimal compression of approximate Euclidean distances
Optimal compression of approximate Euclidean distances Noga Alon 1 Bo az Klartag 2 Abstract Let X be a set of n points of norm at most 1 in the Euclidean space R k, and suppose ε > 0. An ε-distance sketch
More informationLinear Analysis Lecture 5
Linear Analysis Lecture 5 Inner Products and V Let dim V < with inner product,. Choose a basis B and let v, w V have coordinates in F n given by x 1. x n and y 1. y n, respectively. Let A F n n be the
More informationGraph Sparsifiers: A Survey
Graph Sparsifiers: A Survey Nick Harvey UBC Based on work by: Batson, Benczur, de Carli Silva, Fung, Hariharan, Harvey, Karger, Panigrahi, Sato, Spielman, Srivastava and Teng Approximating Dense Objects
More informationThe Chromatic Number of Random Regular Graphs
The Chromatic Number of Random Regular Graphs Dimitris Achlioptas 1 and Cristopher Moore 2 1 Microsoft Research, Redmond, WA 98052, USA optas@microsoft.com 2 University of New Mexico, NM 87131, USA moore@cs.unm.edu
More informationLecture 4. P r[x > ce[x]] 1/c. = ap r[x = a] + a>ce[x] P r[x = a]
U.C. Berkeley CS273: Parallel and Distributed Theory Lecture 4 Professor Satish Rao September 7, 2010 Lecturer: Satish Rao Last revised September 13, 2010 Lecture 4 1 Deviation bounds. Deviation bounds
More informationHardness of Embedding Metric Spaces of Equal Size
Hardness of Embedding Metric Spaces of Equal Size Subhash Khot and Rishi Saket Georgia Institute of Technology {khot,saket}@cc.gatech.edu Abstract. We study the problem embedding an n-point metric space
More informationFunctional Analysis Exercise Class
Functional Analysis Exercise Class Week 9 November 13 November Deadline to hand in the homeworks: your exercise class on week 16 November 20 November Exercises (1) Show that if T B(X, Y ) and S B(Y, Z)
More informationSuccinct Data Structures for Approximating Convex Functions with Applications
Succinct Data Structures for Approximating Convex Functions with Applications Prosenjit Bose, 1 Luc Devroye and Pat Morin 1 1 School of Computer Science, Carleton University, Ottawa, Canada, K1S 5B6, {jit,morin}@cs.carleton.ca
More information