Kernel Density Estimation
|
|
- Priscilla Gallagher
- 6 years ago
- Views:
Transcription
1 EECS 598: Statistical Learning Theory, Winter 2014 Topic 19 Kernel Density Estimation Lecturer: Clayton Scott Scribe: Yun Wei, Yanzhen Deng Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications. They may be distributed outside this class only with the permission of the Instructor. 1 Introduction Let f be a density on R d, i.e. f 0 and f(x)dx = 1. Suppose X 1, X 2,..., X n iid f. Let φ be a function s.t. φ(x)dx = 1, called a kernel, and denote φ σ (x) := 1 σ d φ( x σ ) for σ > 0. σ is called the bandwidth. The kernel density estimator (KDE) is f n (x) := 1 n φ σ (x X i ). Example. 1) Gaussian kernel: φ(x) = (2π) d 2 e x 2 2 2) There are some common kernels like triangle kernel and box kernel. See Figure 1 for their graph in one dimension. 2 L p Space For f : R R and 0 < p <, define f p = ( f(x) p dx) 1 p and L p = {f f p < }. If p 1 and we identify f and g when f g p = 0 (thus defining equivalence classes) then L p is a normed vector space, where the triangle inequality is given by Minkowski s Inequality. For a full development, see [1]. Definition 1 (Convolution). Given f, g, the convolution f g is the function f g(x) = f(y)g(x y)dy = g(y)f(x y)dy. Young s Inequality shows that the convolution of L 1 functions is still an L 1 function. Lemma 1 (Young s Inequality). If f, g L 1, then f g L 1 and f g 1 f 1 g 1. 1
2 2 Figure 1: This is a picture showing some kernels. In addition to the uniform and triangular kernels, the third example is an arbitrary kernel illustrating that a kernel need not be 1) positive, or 2) symmetric. Proof. f g 1 = = = = f g(x) dx f(y)g(x y)dy dx ( f(y)g(x y) dy)dx f(y) ( g(x y) dx)dy (By Tonelli Theorem) f(y) g 1 dy (By substitution: u = x y) = f 1 g 1. We state the next result without proof. Theorem 1 (See Folland, Thm 8.14). Let f L p, and φ L 1 with φ(x)dx = a. Then for any r > 0, f φ r L p and lim r 0 f φ r af p = 0.
3 3 3 L 2 consistency Theorem 2. Let f L 2 be a density, φ L 1 L 2 with φ(x)dx = 1. Assume X 1, X 2,..., X n iid f. If σ 0 and nσ d as n, then f n f 2 i.p. 0. Proof. By the triangle inequality, f n f 2 f n f φ σ 2 + f φ σ f 2. The second term 0 as σ 0, by Theorem 1, since f L 2, φ L 1. The first term converges i.p. to zero according to Lemma 2. Lemma 2. If f is a density, φ L 2, and X 1, X 2,..., X n iid f, then provided nσ d. Proof. Observe f n f φ σ 2 i.p. 0 Pr{ f n f φ σ 2 > ɛ} = Pr{ f n f φ σ 2 2 > ɛ 2 } E{ f n f φ σ 2 2}/ɛ 2 by Markov s Inequality. So it suffices to show E{ f n f φ σ 2 2} 0. Note E is an integral operator, and therefore by Tonelli s Theorem we can interchange the order of integration: E{ f n f φ σ 2 2} = E{( f n (x) f φ σ (x)) 2 }dx. Write f n (x) f φ σ (x) = 1 n Z i, where Z i = φ σ (x X i ) f φ σ (x). Note that Z i are iid and E(Z i ) = 0 because Eφ σ (x X i ) = φ σ (x x i )f(x i )dx i = f φ σ (x). The variance of Z i is E(Z 2 i ) = Var(φ σ (x X i )) = E{(φ σ (x X i )) 2 } [E{φ σ (x X i )}] 2 E{(φ σ (x X i )) 2 } = f(y)φ σ (x y) 2 dy = f φ 2 σ(x) = 1 σ d f (φ2 ) σ (x),
4 4 where the last step follows from the fact φ 2 σ(x) = [ 1 σ d φ( x σ )]2 = 1 σ d [ 1 σ d φ 2 ( x σ )] = 1 σ d (φ 2 ) σ. Thus E{( 1 n Z i ) 2 } = 1 n E{Z2 1} 1 nσ d f (φ2 ) σ (x) and therefore E{( f n (x) f φ σ (x)) 2 }dx 1 nσ d f (φ 2 ) σ (x)dx = 1 nσ d f (φ2 ) σ (x) 1 1 nσ d f 1 (φ 2 ) σ 1 (by Young s Inequality) = 1 nσ d φ 2 2 ( f 1 = 1) 0, since nσ d and φ L 2. Remark. (1) The condition f L 2 excludes certain densities such as where 1 2 < r < 1. (2) φ L 2 is satisfied by all common kernels. f(x) = 1 1 r x r, 0 < x < 1, (3) Recall φ need not be symmetric w.r.t. the origin. Thus, the consistency result holds for where φ(x) = 1 {x B(x0,r)}, B(x 0, r) = {x : x x 0 2 r}, r is s.t. φ(x)dx = 1, and x 0 = (0, 0,, 0, ). This may seem bizarre, but as an exercise you are asked to make sense of this example. 4 L 1 -Consistency In this section we will show that the L 1 error converges to 0 in probability. To keep things simpler, we will assume f has compact support, although this is not necessary for L 1 consistency. Theorem 3. If f is a density with compact support, φ L 1 s.t. then f i.p. n f 1 0, provided that σ 0 and nσ d 0 and n. φ(x)dx = 1, and X1, X 2,..., X n i.i.d. f, Proof. Note that f n f 1 f n f φ σ 1 + f φ σ f 1. By Theorem 1, we know that f φ σ f 1 0, so it remains to show convergence to zero of f n f φ σ 1.
5 5 Let C c = {g : R d R g is bounded and has compact support}. It is a well-known fact in analysis [1] that C c is dense in L 1. Thus for any fixed ɛ > 0, we can take ψ C c s.t. φ ψ 1 < ɛ. Denote f c n(x) = 1 n n ψ σ(x X i ). Note that By Young s Inequality, The first term is bounded by f n f φ σ 1 f n f c n 1 + f c n f ψ σ 1 + f ψ σ f φ σ 1. f ψ σ f φ σ 1 = f (ψ σ φ σ ) 1 f 1 ψ σ φ σ = ψ φ 1 < ɛ. f n f c n 1 1 n ψ σ (x X i ) φ σ (x X i ) 1 < ɛ. Since ɛ is arbitrary, we only need to prove that f n c f ψ σ 1 0 i.p. Denote by S f and S ψ the supports of f and ψ, respectively. We know that S f and S ψ are both compact sets and S ψσ is also compact and shrinks as σ 0. Thus, f n c f ψ σ 1 = f n c f ψ σ dx S f S ψσ = f n c f ψ σ dx (for σ 1) S f S ψ = f n c f ψ σ 1 Sf S ψ dx f c n f ψ σ 2 1 Sf S ψ 2. (Hölder s Inequality) The second equality holds when σ < 1, which implies S f S ψσ S f S ψ. Since ψ is in L 1 and bounded with compact support, it is also in L 2. Thus by Lemma 2, f n c f ψ σ 2 0 i.p. Now 1 Sf S ψ 2 is the square root of the volume of a compact set and thus is finite. Therefore f n c f ψ σ 1 0 i.p. Remark. The reason that we care about L 1 error is the following equality called Scheffe s Identity: if f, g are densities and B is the set of Borel sets, then: f g 1 = (f g)(x)dx (g f)(x)dx f<g = (f g)(x)dx [ (g f)(x)dx (g f)(x)dx] = 2 (f g)(x)dx = 2 sup f(x)dx g(x)dx B B B B Scheffe s Identity shows that small L 1 error leads to accurate probability estimation. 5 Strong Consistency If we add the constraint that the kernel be nonnegative, then weak L 1 consistency implies strong L 1 consistency.
6 6 Theorem 4. Assume φ 0 and φ(x)dx = 1. If X 1, X 2,..., X n i.i.d. f, then f n f 1 0 i.p. implies f n f 1 0 a.s. Proof. Let S = (X 1,..., X n ) and S i = (X 1,..., X i 1, X i, X i+1,..., X n ). Write f n = f n,s, using the new subscript to indicate the sample. Denote φ n (S) = f n,s f 1. Then φ n (S) φ n (S i) f n,s f n,s i 1 (reverse triangle inequality) = 1 φ σ (x X i ) φ σ (x X n i) dx 1 φ σ (x X i ) dx + φ σ (x X n i) dx By the bounded difference inequality, = 2. (φ nonnegative) n Pr(φ n (S) E[φ n (S)] ɛ) e nɛ2 /2. Fix ɛ > 0. By weak consistency, N s.t. n N Eφ n (S) < ɛ 2. Then for n N, This upper bound decrease geometrically. Therefore and Borel-Cantelli implies φ n (S) 0 a.s. Exercises Pr(φ n (S) ɛ) Pr(φ n (S) Eφ n (S) ɛ/2) e nɛ2 /8. Pr(φ n (S) ɛ) < n=1 1. Make sense of the third remark after the proof of L 2 consistency. 2. What does Bernstein s inequality imply about 1 n Zi in the proof of Lemma 2? Is this observation useful in any way? 3. Remove the assumption in Theorem 3 that f has compact support. References [1] Gerald B. Folland, Real Analysis: Modern Techniques and Their Applications, Wiley, 1999
EECS 598: Statistical Learning Theory, Winter 2014 Topic 11. Kernels
EECS 598: Statistical Learning Theory, Winter 2014 Topic 11 Kernels Lecturer: Clayton Scott Scribe: Jun Guo, Soumik Chatterjee Disclaimer: These notes have not been subjected to the usual scrutiny reserved
More informationLebesgue Integration on R n
Lebesgue Integration on R n The treatment here is based loosely on that of Jones, Lebesgue Integration on Euclidean Space We give an overview from the perspective of a user of the theory Riemann integration
More informationFall f(x)g(x) dx. The starting place for the theory of Fourier series is that the family of functions {e inx } n= is orthonormal, that is
18.103 Fall 2013 1. Fourier Series, Part 1. We will consider several function spaces during our study of Fourier series. When we talk about L p ((, π)), it will be convenient to include the factor 1/ in
More informationTHE STONE-WEIERSTRASS THEOREM AND ITS APPLICATIONS TO L 2 SPACES
THE STONE-WEIERSTRASS THEOREM AND ITS APPLICATIONS TO L 2 SPACES PHILIP GADDY Abstract. Throughout the course of this paper, we will first prove the Stone- Weierstrass Theroem, after providing some initial
More informationLebesgue s Differentiation Theorem via Maximal Functions
Lebesgue s Differentiation Theorem via Maximal Functions Parth Soneji LMU München Hütteseminar, December 2013 Parth Soneji Lebesgue s Differentiation Theorem via Maximal Functions 1/12 Philosophy behind
More informationMATH MEASURE THEORY AND FOURIER ANALYSIS. Contents
MATH 3969 - MEASURE THEORY AND FOURIER ANALYSIS ANDREW TULLOCH Contents 1. Measure Theory 2 1.1. Properties of Measures 3 1.2. Constructing σ-algebras and measures 3 1.3. Properties of the Lebesgue measure
More informationMath 328 Course Notes
Math 328 Course Notes Ian Robertson March 3, 2006 3 Properties of C[0, 1]: Sup-norm and Completeness In this chapter we are going to examine the vector space of all continuous functions defined on the
More informationFunctional Analysis, Stein-Shakarchi Chapter 1
Functional Analysis, Stein-Shakarchi Chapter 1 L p spaces and Banach Spaces Yung-Hsiang Huang 018.05.1 Abstract Many problems are cited to my solution files for Folland [4] and Rudin [6] post here. 1 Exercises
More informationChapter 8 Integral Operators
Chapter 8 Integral Operators In our development of metrics, norms, inner products, and operator theory in Chapters 1 7 we only tangentially considered topics that involved the use of Lebesgue measure,
More informationFunctional Analysis Exercise Class
Functional Analysis Exercise Class Week 9 November 13 November Deadline to hand in the homeworks: your exercise class on week 16 November 20 November Exercises (1) Show that if T B(X, Y ) and S B(Y, Z)
More informationCLASSICAL PROBABILITY MODES OF CONVERGENCE AND INEQUALITIES
CLASSICAL PROBABILITY 2008 2. MODES OF CONVERGENCE AND INEQUALITIES JOHN MORIARTY In many interesting and important situations, the object of interest is influenced by many random factors. If we can construct
More informationBasic Properties of Metric and Normed Spaces
Basic Properties of Metric and Normed Spaces Computational and Metric Geometry Instructor: Yury Makarychev The second part of this course is about metric geometry. We will study metric spaces, low distortion
More informationDefinition 6.1. A metric space (X, d) is complete if every Cauchy sequence tends to a limit in X.
Chapter 6 Completeness Lecture 18 Recall from Definition 2.22 that a Cauchy sequence in (X, d) is a sequence whose terms get closer and closer together, without any limit being specified. In the Euclidean
More informationMTH 404: Measure and Integration
MTH 404: Measure and Integration Semester 2, 2012-2013 Dr. Prahlad Vaidyanathan Contents I. Introduction....................................... 3 1. Motivation................................... 3 2. The
More informationThe reference [Ho17] refers to the course lecture notes by Ilkka Holopainen.
Department of Mathematics and Statistics Real Analysis I, Fall 207 Solutions to Exercise 6 (6 pages) riikka.schroderus at helsinki.fi Note. The course can be passed by an exam. The first possible exam
More information0.1 Uniform integrability
Copyright c 2009 by Karl Sigman 0.1 Uniform integrability Given a sequence of rvs {X n } for which it is known apriori that X n X, n, wp1. for some r.v. X, it is of great importance in many applications
More information10-704: Information Processing and Learning Fall Lecture 9: Sept 28
10-704: Information Processing and Learning Fall 2016 Lecturer: Siheng Chen Lecture 9: Sept 28 Note: These notes are based on scribed notes from Spring15 offering of this course. LaTeX template courtesy
More informationTheorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension. n=1
Chapter 2 Probability measures 1. Existence Theorem 2.1 (Caratheodory). A (countably additive) probability measure on a field has an extension to the generated σ-field Proof of Theorem 2.1. Let F 0 be
More information4 Riesz Kernels. Since the functions k i (ξ) = ξ i. are bounded functions it is clear that R
4 Riesz Kernels. A natural generalization of the Hilbert transform to higher dimension is mutiplication of the Fourier Transform by homogeneous functions of degree 0, the simplest ones being R i f(ξ) =
More informationL p Spaces and Convexity
L p Spaces and Convexity These notes largely follow the treatments in Royden, Real Analysis, and Rudin, Real & Complex Analysis. 1. Convex functions Let I R be an interval. For I open, we say a function
More informationMATH 411 NOTES (UNDER CONSTRUCTION)
MATH 411 NOTES (NDE CONSTCTION 1. Notes on compact sets. This is similar to ideas you learned in Math 410, except open sets had not yet been defined. Definition 1.1. K n is compact if for every covering
More information1 Topology Definition of a topology Basis (Base) of a topology The subspace topology & the product topology on X Y 3
Index Page 1 Topology 2 1.1 Definition of a topology 2 1.2 Basis (Base) of a topology 2 1.3 The subspace topology & the product topology on X Y 3 1.4 Basic topology concepts: limit points, closed sets,
More informationMath 209B Homework 2
Math 29B Homework 2 Edward Burkard Note: All vector spaces are over the field F = R or C 4.6. Two Compactness Theorems. 4. Point Set Topology Exercise 6 The product of countably many sequentally compact
More informationProblem set 1, Real Analysis I, Spring, 2015.
Problem set 1, Real Analysis I, Spring, 015. (1) Let f n : D R be a sequence of functions with domain D R n. Recall that f n f uniformly if and only if for all ɛ > 0, there is an N = N(ɛ) so that if n
More information{σ x >t}p x. (σ x >t)=e at.
3.11. EXERCISES 121 3.11 Exercises Exercise 3.1 Consider the Ornstein Uhlenbeck process in example 3.1.7(B). Show that the defined process is a Markov process which converges in distribution to an N(0,σ
More informationLecture 18: March 15
CS71 Randomness & Computation Spring 018 Instructor: Alistair Sinclair Lecture 18: March 15 Disclaimer: These notes have not been subjected to the usual scrutiny accorded to formal publications. They may
More informationFall TMA4145 Linear Methods. Solutions to exercise set 9. 1 Let X be a Hilbert space and T a bounded linear operator on X.
TMA445 Linear Methods Fall 26 Norwegian University of Science and Technology Department of Mathematical Sciences Solutions to exercise set 9 Let X be a Hilbert space and T a bounded linear operator on
More informationMath The Laplacian. 1 Green s Identities, Fundamental Solution
Math. 209 The Laplacian Green s Identities, Fundamental Solution Let be a bounded open set in R n, n 2, with smooth boundary. The fact that the boundary is smooth means that at each point x the external
More informationMetric Spaces and Topology
Chapter 2 Metric Spaces and Topology From an engineering perspective, the most important way to construct a topology on a set is to define the topology in terms of a metric on the set. This approach underlies
More informationMATH 426, TOPOLOGY. p 1.
MATH 426, TOPOLOGY THE p-norms In this document we assume an extended real line, where is an element greater than all real numbers; the interval notation [1, ] will be used to mean [1, ) { }. 1. THE p
More informationSOLUTIONS TO ASSIGNMENT 2 - MATH 355. with c > 3. m(n c ) < δ. f(t) t. g(x)dx =
SOLUTIONS TO ASSIGNMENT 2 - MATH 355 Problem. ecall ha, B n {ω [, ] : S n (ω) > nɛ n }, and S n (ω) N {ω [, ] : lim }, n n m(b n ) 3 n 2 ɛ 4. We wan o show ha m(n c ). Le δ >. We can pick ɛ 4 n c n wih
More informationImmerse Metric Space Homework
Immerse Metric Space Homework (Exercises -2). In R n, define d(x, y) = x y +... + x n y n. Show that d is a metric that induces the usual topology. Sketch the basis elements when n = 2. Solution: Steps
More informationL p Functions. Given a measure space (X, µ) and a real number p [1, ), recall that the L p -norm of a measurable function f : X R is defined by
L p Functions Given a measure space (, µ) and a real number p [, ), recall that the L p -norm of a measurable function f : R is defined by f p = ( ) /p f p dµ Note that the L p -norm of a function f may
More information2. Function spaces and approximation
2.1 2. Function spaces and approximation 2.1. The space of test functions. Notation and prerequisites are collected in Appendix A. Let Ω be an open subset of R n. The space C0 (Ω), consisting of the C
More informationReview of measure theory
209: Honors nalysis in R n Review of measure theory 1 Outer measure, measure, measurable sets Definition 1 Let X be a set. nonempty family R of subsets of X is a ring if, B R B R and, B R B R hold. bove,
More informationCalibrated Surrogate Losses
EECS 598: Statistical Learning Theory, Winter 2014 Topic 14 Calibrated Surrogate Losses Lecturer: Clayton Scott Scribe: Efrén Cruz Cortés Disclaimer: These notes have not been subjected to the usual scrutiny
More informationThe Heine-Borel and Arzela-Ascoli Theorems
The Heine-Borel and Arzela-Ascoli Theorems David Jekel October 29, 2016 This paper explains two important results about compactness, the Heine- Borel theorem and the Arzela-Ascoli theorem. We prove them
More information7: FOURIER SERIES STEVEN HEILMAN
7: FOURIER SERIES STEVE HEILMA Contents 1. Review 1 2. Introduction 1 3. Periodic Functions 2 4. Inner Products on Periodic Functions 3 5. Trigonometric Polynomials 5 6. Periodic Convolutions 7 7. Fourier
More informationMost Continuous Functions are Nowhere Differentiable
Most Continuous Functions are Nowhere Differentiable Spring 2004 The Space of Continuous Functions Let K = [0, 1] and let C(K) be the set of all continuous functions f : K R. Definition 1 For f C(K) we
More information10-704: Information Processing and Learning Spring Lecture 8: Feb 5
10-704: Information Processing and Learning Spring 2015 Lecture 8: Feb 5 Lecturer: Aarti Singh Scribe: Siheng Chen Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal
More informationd(x n, x) d(x n, x nk ) + d(x nk, x) where we chose any fixed k > N
Problem 1. Let f : A R R have the property that for every x A, there exists ɛ > 0 such that f(t) > ɛ if t (x ɛ, x + ɛ) A. If the set A is compact, prove there exists c > 0 such that f(x) > c for all x
More informationIntroduction to Real Analysis Alternative Chapter 1
Christopher Heil Introduction to Real Analysis Alternative Chapter 1 A Primer on Norms and Banach Spaces Last Updated: March 10, 2018 c 2018 by Christopher Heil Chapter 1 A Primer on Norms and Banach Spaces
More informationTHEOREMS, ETC., FOR MATH 515
THEOREMS, ETC., FOR MATH 515 Proposition 1 (=comment on page 17). If A is an algebra, then any finite union or finite intersection of sets in A is also in A. Proposition 2 (=Proposition 1.1). For every
More informationg(x) = P (y) Proof. This is true for n = 0. Assume by the inductive hypothesis that g (n) (0) = 0 for some n. Compute g (n) (h) g (n) (0)
Mollifiers and Smooth Functions We say a function f from C is C (or simply smooth) if all its derivatives to every order exist at every point of. For f : C, we say f is C if all partial derivatives to
More informationbe the set of complex valued 2π-periodic functions f on R such that
. Fourier series. Definition.. Given a real number P, we say a complex valued function f on R is P -periodic if f(x + P ) f(x) for all x R. We let be the set of complex valued -periodic functions f on
More information1 Fourier Integrals on L 2 (R) and L 1 (R).
18.103 Fall 2013 1 Fourier Integrals on L 2 () and L 1 (). The first part of these notes cover 3.5 of AG, without proofs. When we get to things not covered in the book, we will start giving proofs. The
More informationMath 115 ( ) Yum-Tong Siu 1. Derivation of the Poisson Kernel by Fourier Series and Convolution
Math 5 (006-007 Yum-Tong Siu. Derivation of the Poisson Kernel by Fourier Series and Convolution We are going to give a second derivation of the Poisson kernel by using Fourier series and convolution.
More information10-704: Information Processing and Learning Fall Lecture 24: Dec 7
0-704: Information Processing and Learning Fall 206 Lecturer: Aarti Singh Lecture 24: Dec 7 Note: These notes are based on scribed notes from Spring5 offering of this course. LaTeX template courtesy of
More informationAnother Riesz Representation Theorem
Another Riesz Representation Theorem In these notes we prove (one version of) a theorem known as the Riesz Representation Theorem. Some people also call it the Riesz Markov Theorem. It expresses positive
More information4 Expectation & the Lebesgue Theorems
STA 205: Probability & Measure Theory Robert L. Wolpert 4 Expectation & the Lebesgue Theorems Let X and {X n : n N} be random variables on a probability space (Ω,F,P). If X n (ω) X(ω) for each ω Ω, does
More information1.5 Approximate Identities
38 1 The Fourier Transform on L 1 (R) which are dense subspaces of L p (R). On these domains, P : D P L p (R) and M : D M L p (R). Show, however, that P and M are unbounded even when restricted to these
More informationNotes on uniform convergence
Notes on uniform convergence Erik Wahlén erik.wahlen@math.lu.se January 17, 2012 1 Numerical sequences We begin by recalling some properties of numerical sequences. By a numerical sequence we simply mean
More informationSOLUTIONS TO HOMEWORK ASSIGNMENT 4
SOLUTIONS TO HOMEWOK ASSIGNMENT 4 Exercise. A criterion for the image under the Hilbert transform to belong to L Let φ S be given. Show that Hφ L if and only if φx dx = 0. Solution: Suppose first that
More information2 (Bonus). Let A X consist of points (x, y) such that either x or y is a rational number. Is A measurable? What is its Lebesgue measure?
MA 645-4A (Real Analysis), Dr. Chernov Homework assignment 1 (Due 9/5). Prove that every countable set A is measurable and µ(a) = 0. 2 (Bonus). Let A consist of points (x, y) such that either x or y is
More informationMeasure and Integration: Solutions of CW2
Measure and Integration: s of CW2 Fall 206 [G. Holzegel] December 9, 206 Problem of Sheet 5 a) Left (f n ) and (g n ) be sequences of integrable functions with f n (x) f (x) and g n (x) g (x) for almost
More informationLecture Notes on Metric Spaces
Lecture Notes on Metric Spaces Math 117: Summer 2007 John Douglas Moore Our goal of these notes is to explain a few facts regarding metric spaces not included in the first few chapters of the text [1],
More informationMath 172 Problem Set 5 Solutions
Math 172 Problem Set 5 Solutions 2.4 Let E = {(t, x : < x b, x t b}. To prove integrability of g, first observe that b b b f(t b b g(x dx = dt t dx f(t t dtdx. x Next note that f(t/t χ E is a measurable
More informationh(x) lim H(x) = lim Since h is nondecreasing then h(x) 0 for all x, and if h is discontinuous at a point x then H(x) > 0. Denote
Real Variables, Fall 4 Problem set 4 Solution suggestions Exercise. Let f be of bounded variation on [a, b]. Show that for each c (a, b), lim x c f(x) and lim x c f(x) exist. Prove that a monotone function
More informationAnalysis Comprehensive Exam Questions Fall F(x) = 1 x. f(t)dt. t 1 2. tf 2 (t)dt. and g(t, x) = 2 t. 2 t
Analysis Comprehensive Exam Questions Fall 2. Let f L 2 (, ) be given. (a) Prove that ( x 2 f(t) dt) 2 x x t f(t) 2 dt. (b) Given part (a), prove that F L 2 (, ) 2 f L 2 (, ), where F(x) = x (a) Using
More informationSingular Integrals. 1 Calderon-Zygmund decomposition
Singular Integrals Analysis III Calderon-Zygmund decomposition Let f be an integrable function f dx 0, f = g + b with g Cα almost everywhere, with b
More informationEmpirical Processes: General Weak Convergence Theory
Empirical Processes: General Weak Convergence Theory Moulinath Banerjee May 18, 2010 1 Extended Weak Convergence The lack of measurability of the empirical process with respect to the sigma-field generated
More informationProblem Set 5. 2 n k. Then a nk (x) = 1+( 1)k
Problem Set 5 1. (Folland 2.43) For x [, 1), let 1 a n (x)2 n (a n (x) = or 1) be the base-2 expansion of x. (If x is a dyadic rational, choose the expansion such that a n (x) = for large n.) Then the
More informationContinuity. Matt Rosenzweig
Continuity Matt Rosenzweig Contents 1 Continuity 1 1.1 Rudin Chapter 4 Exercises........................................ 1 1.1.1 Exercise 1............................................. 1 1.1.2 Exercise
More informationLecture I: Asymptotics for large GUE random matrices
Lecture I: Asymptotics for large GUE random matrices Steen Thorbjørnsen, University of Aarhus andom Matrices Definition. Let (Ω, F, P) be a probability space and let n be a positive integer. Then a random
More informationSome Background Material
Chapter 1 Some Background Material In the first chapter, we present a quick review of elementary - but important - material as a way of dipping our toes in the water. This chapter also introduces important
More informationMATH 202B - Problem Set 5
MATH 202B - Problem Set 5 Walid Krichene (23265217) March 6, 2013 (5.1) Show that there exists a continuous function F : [0, 1] R which is monotonic on no interval of positive length. proof We know there
More information4th Preparation Sheet - Solutions
Prof. Dr. Rainer Dahlhaus Probability Theory Summer term 017 4th Preparation Sheet - Solutions Remark: Throughout the exercise sheet we use the two equivalent definitions of separability of a metric space
More informationLecture 3: Expected Value. These integrals are taken over all of Ω. If we wish to integrate over a measurable subset A Ω, we will write
Lecture 3: Expected Value 1.) Definitions. If X 0 is a random variable on (Ω, F, P), then we define its expected value to be EX = XdP. Notice that this quantity may be. For general X, we say that EX exists
More informationUseful Probability Theorems
Useful Probability Theorems Shiu-Tang Li Finished: March 23, 2013 Last updated: November 2, 2013 1 Convergence in distribution Theorem 1.1. TFAE: (i) µ n µ, µ n, µ are probability measures. (ii) F n (x)
More information3. (a) What is a simple function? What is an integrable function? How is f dµ defined? Define it first
Math 632/6321: Theory of Functions of a Real Variable Sample Preinary Exam Questions 1. Let (, M, µ) be a measure space. (a) Prove that if µ() < and if 1 p < q
More informationLinear Analysis Lecture 5
Linear Analysis Lecture 5 Inner Products and V Let dim V < with inner product,. Choose a basis B and let v, w V have coordinates in F n given by x 1. x n and y 1. y n, respectively. Let A F n n be the
More informationLecture 6: September 12
10-725: Optimization Fall 2013 Lecture 6: September 12 Lecturer: Ryan Tibshirani Scribes: Micol Marchetti-Bowick Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have not
More informationFourier Transform & Sobolev Spaces
Fourier Transform & Sobolev Spaces Michael Reiter, Arthur Schuster Summer Term 2008 Abstract We introduce the concept of weak derivative that allows us to define new interesting Hilbert spaces the Sobolev
More informationSobolev spaces. May 18
Sobolev spaces May 18 2015 1 Weak derivatives The purpose of these notes is to give a very basic introduction to Sobolev spaces. More extensive treatments can e.g. be found in the classical references
More informationMetric Spaces. Exercises Fall 2017 Lecturer: Viveka Erlandsson. Written by M.van den Berg
Metric Spaces Exercises Fall 2017 Lecturer: Viveka Erlandsson Written by M.van den Berg School of Mathematics University of Bristol BS8 1TW Bristol, UK 1 Exercises. 1. Let X be a non-empty set, and suppose
More information9 Brownian Motion: Construction
9 Brownian Motion: Construction 9.1 Definition and Heuristics The central limit theorem states that the standard Gaussian distribution arises as the weak limit of the rescaled partial sums S n / p n of
More informationFunctional Analysis, Math 7320 Lecture Notes from August taken by Yaofeng Su
Functional Analysis, Math 7320 Lecture Notes from August 30 2016 taken by Yaofeng Su 1 Essentials of Topology 1.1 Continuity Next we recall a stronger notion of continuity: 1.1.1 Definition. Let (X, d
More informationAdvanced Analysis Qualifying Examination Department of Mathematics and Statistics University of Massachusetts. Tuesday, January 16th, 2018
NAME: Advanced Analysis Qualifying Examination Department of Mathematics and Statistics University of Massachusetts Tuesday, January 16th, 2018 Instructions 1. This exam consists of eight (8) problems
More informationThe main results about probability measures are the following two facts:
Chapter 2 Probability measures The main results about probability measures are the following two facts: Theorem 2.1 (extension). If P is a (continuous) probability measure on a field F 0 then it has a
More informationMATH 51H Section 4. October 16, Recall what it means for a function between metric spaces to be continuous:
MATH 51H Section 4 October 16, 2015 1 Continuity Recall what it means for a function between metric spaces to be continuous: Definition. Let (X, d X ), (Y, d Y ) be metric spaces. A function f : X Y is
More informationWORKSHEET FOR THE PUTNAM COMPETITION -REAL ANALYSIS- lim
WORKSHEET FOR THE PUTNAM COMPETITION -REAL ANALYSIS- INSTRUCTOR: CEZAR LUPU Problem Let < x < and x n+ = x n ( x n ), n =,, 3, Show that nx n = Putnam B3, 966 Question? Problem E 334 from the American
More informationAnalysis IV : Assignment 3 Solutions John Toth, Winter ,...). In particular for every fixed m N the sequence (u (n)
Analysis IV : Assignment 3 Solutions John Toth, Winter 203 Exercise (l 2 (Z), 2 ) is a complete and separable Hilbert space. Proof Let {u (n) } n N be a Cauchy sequence. Say u (n) = (..., u 2, (n) u (n),
More information3 (Due ). Let A X consist of points (x, y) such that either x or y is a rational number. Is A measurable? What is its Lebesgue measure?
MA 645-4A (Real Analysis), Dr. Chernov Homework assignment 1 (Due ). Show that the open disk x 2 + y 2 < 1 is a countable union of planar elementary sets. Show that the closed disk x 2 + y 2 1 is a countable
More informationIndeed, the family is still orthogonal if we consider a complex valued inner product ( or an inner product on complex vector space)
Fourier series of complex valued functions Suppose now f is a piecewise continuous complex valued function on [, π], that is f(x) = u(x)+iv(x) such that both u and v are real valued piecewise continuous
More informationLecture 1. 1, 0 x < 1 2 1, 2. x < 1, 0, elsewhere. 1
0 - - -3 Lecture Introductory mathematical ideas The term wavelet, meaning literally little wave, originated in the early 980s in its French version ondelette in the work of Morlet and some French seismologists.
More informationMA677 Assignment #3 Morgan Schreffler Due 09/19/12 Exercise 1 Using Hölder s inequality, prove Minkowski s inequality for f, g L p (R d ), p 1:
Exercise 1 Using Hölder s inequality, prove Minkowski s inequality for f, g L p (R d ), p 1: f + g p f p + g p. Proof. If f, g L p (R d ), then since f(x) + g(x) max {f(x), g(x)}, we have f(x) + g(x) p
More informationRiesz Representation Theorems
Chapter 6 Riesz Representation Theorems 6.1 Dual Spaces Definition 6.1.1. Let V and W be vector spaces over R. We let L(V, W ) = {T : V W T is linear}. The space L(V, R) is denoted by V and elements of
More informationSOLUTIONS OF SELECTED PROBLEMS
SOLUTIONS OF SELECTED PROBLEMS Problem 36, p. 63 If µ(e n < and χ En f in L, then f is a.e. equal to a characteristic function of a measurable set. Solution: By Corollary.3, there esists a subsequence
More information1/12/05: sec 3.1 and my article: How good is the Lebesgue measure?, Math. Intelligencer 11(2) (1989),
Real Analysis 2, Math 651, Spring 2005 April 26, 2005 1 Real Analysis 2, Math 651, Spring 2005 Krzysztof Chris Ciesielski 1/12/05: sec 3.1 and my article: How good is the Lebesgue measure?, Math. Intelligencer
More information(1) Consider the space S consisting of all continuous real-valued functions on the closed interval [0, 1]. For f, g S, define
Homework, Real Analysis I, Fall, 2010. (1) Consider the space S consisting of all continuous real-valued functions on the closed interval [0, 1]. For f, g S, define ρ(f, g) = 1 0 f(x) g(x) dx. Show that
More information17.1 Hyperplanes and Linear Forms
This is page 530 Printer: Opaque this 17 Appendix 17.1 Hyperplanes and Linear Forms Given a vector space E over a field K, a linear map f: E K is called a linear form. The set of all linear forms f: E
More informationAnalysis III Theorems, Propositions & Lemmas... Oh My!
Analysis III Theorems, Propositions & Lemmas... Oh My! Rob Gibson October 25, 2010 Proposition 1. If x = (x 1, x 2,...), y = (y 1, y 2,...), then is a distance. ( d(x, y) = x k y k p Proposition 2. In
More informationHOW TO LOOK AT MINKOWSKI S THEOREM
HOW TO LOOK AT MINKOWSKI S THEOREM COSMIN POHOATA Abstract. In this paper we will discuss a few ways of thinking about Minkowski s lattice point theorem. The Minkowski s Lattice Point Theorem essentially
More informationProblem set 5, Real Analysis I, Spring, otherwise. (a) Verify that f is integrable. Solution: Compute since f is even, 1 x (log 1/ x ) 2 dx 1
Problem set 5, Real Analysis I, Spring, 25. (5) Consider the function on R defined by f(x) { x (log / x ) 2 if x /2, otherwise. (a) Verify that f is integrable. Solution: Compute since f is even, R f /2
More informationREAL ANALYSIS II HOMEWORK 3. Conway, Page 49
REAL ANALYSIS II HOMEWORK 3 CİHAN BAHRAN Conway, Page 49 3. Let K and k be as in Proposition 4.7 and suppose that k(x, y) k(y, x). Show that K is self-adjoint and if {µ n } are the eigenvalues of K, each
More informationSelçuk Demir WS 2017 Functional Analysis Homework Sheet
Selçuk Demir WS 2017 Functional Analysis Homework Sheet 1. Let M be a metric space. If A M is non-empty, we say that A is bounded iff diam(a) = sup{d(x, y) : x.y A} exists. Show that A is bounded iff there
More informationAustin Mohr Math 704 Homework 6
Austin Mohr Math 704 Homework 6 Problem 1 Integrability of f on R does not necessarily imply the convergence of f(x) to 0 as x. a. There exists a positive continuous function f on R so that f is integrable
More informationExercise 1. Let f be a nonnegative measurable function. Show that. where ϕ is taken over all simple functions with ϕ f. k 1.
Real Variables, Fall 2014 Problem set 3 Solution suggestions xercise 1. Let f be a nonnegative measurable function. Show that f = sup ϕ, where ϕ is taken over all simple functions with ϕ f. For each n
More informationFunctional Analysis Exercise Class
Functional Analysis Exercise Class Week 2 November 6 November Deadline to hand in the homeworks: your exercise class on week 9 November 13 November Exercises (1) Let X be the following space of piecewise
More information10-704: Information Processing and Learning Fall Lecture 21: Nov 14. sup
0-704: Information Processing and Learning Fall 206 Lecturer: Aarti Singh Lecture 2: Nov 4 Note: hese notes are based on scribed notes from Spring5 offering of this course LaeX template courtesy of UC
More information