Week 10. f2 j=2 2 j k ; j; k 2 Zg is an orthonormal basis for L 2 (R). This function is called mother wavelet, which can be often constructed
|
|
- Evelyn Norton
- 6 years ago
- Views:
Transcription
1 Wee 0 A Itroductio to Wavelet regressio. De itio: Wavelet is a fuctio such that f j= j ; j; Zg is a orthoormal basis for L (R). This fuctio is called mother wavelet, which ca be ofte costructed from father wavelet '. The father wavelet ' is ot a wavelet, but we ca costruct wavelets from it, so it is equally importat as mother wavelet. Example: Haar wavelet (A. Haar, Math. A. (90)) ad (x) = x [0; =) x [=; ) j= j j= x = j ; j + (j+) j= x j + (j+) ; j + j. Let ' (t) is the idicator fuctio over the iterval [0; ). Let s de e V j = spa ' j = j= ' j o, Z Sice ' (x) = ' (x ) + ' (x), the followig four properties are satis ed (i)... V V V 0 V V :::; (ii) f (x) V j () f (x) V j+ ; (iii) [ jz V j = L (R), (iv) there is a fuctio ' such that f' ( ) ; Zg is a orthoormal basis for V 0. A equivalet form of property (iii): Recall that bf () = Z f (x) exp ( ix) dx It ca be show that, for ay ' the property (iii) is equivalet to b' (0) 6= 0 ad jb'j is cotiuous at 0. A setch of the proof is as follows: First, [ jz V j is ivariace uder all traslatios. Secod, if g is orthogoal to [ jz V j, which implies g (x) is orthogoal to ' j (x + t) for all t, the Placherel formula implies bg () b' j = 0 a.s.. We ow b' 6= 0 aroud 0. Let j!, we the coclude bg () = 0 a.s.. From ' to : Observe that (x) = ' (x) ' (x )
2 ad If we de e h'; i = 0. W j = spa j= j o, Z the ad more geerally We also see that JL j= i other words,f j = j= j L (R) : V 0 L W0 = V V j L Wj = V j+ W j = V J+! L (R), J!,, j; Zg is a orthoormal basis for. Multiresolutio aalysis (MRA) a geeral framewor to costruct wavelet fuctios More geerally, if there is ' such that f' ( ) ; Zg is a orthoormal system, ad x p ' = h ' (x ) adjb'j is cotiuous at 0 with b' (0) 6= 0. Let s de e V j = spa j= ' j o, Z The the followig four properties are satis ed i)... V V V 0 V V :::; ii) f (x) V j () f (x) V j+ ; iii) [ jz V j = L (R) iv) there is a fuctio ' such that f' ( ) ; Zg is a orthoormal basis for V 0. This is called a MRA i L (R). It is easy to see property (iv) is equivalet to Z ;0 = ' (x) ' (x ) dx = Z b' () e i d = Z for all, i.e., P l= jb' ( + l)j = a.s.. From ' to : There is a sequece fh g such that x p ' = h ' (x ) 0 l= jb' ( + l)j e i d the b' () = p b' () b h () = M p b' () h exp (i).
3 It is easy to see b h (0) = p ad b h () + b h ( + ) = which is due to the idetity P l= jb' ( + l)j =. If satis es x p = g ' (x ) the De e i.e., b () = p b' () bg () = M p b' () g exp (i). g = ( ) h bg () = ( ) h exp (i) = ( ) h exp ( i ( ) ) = e i b h ( + ); the which implies f ( l= b ( + l) = ) ; Zg is a orthoormal system, ad bg () b h () + bg ( + ) b h ( + ) = 0 which implies spa f ( ) ; Zg? spa f' ( ) ; Zg because De i b () ; b' () E ad it is easy to see More geerally ad = D e i b' () bg () ; b' () b E h () = Z h e i bg () b h () + bg ( + ) b i h ( + ) d = 0. 0 spa f ( ) ; Zg L spa f' ( ) ; Zg = V. JL j= where W j = spaf j = j= j V j L Wj = V j+ W j = V J+! L (R), J!, ; Zg. 3
4 Costruct ' (Meyer, Mallat): If b h () is periodic, C ear = 0, ad (i) b h (0) = p ad b h () + b h ( + ) = (ii) if [ =;=] b h () > 0, the Q b' () = p b h j is the Fourier trasformatio of a scalig fuctio ' L that geerates a MRA. 3. Wavelets o the iterval. Haar wavelet ca be modi ed to be a orthogoal bases for L (0; ) ' (x) ; j= j x ; j = 0; ; ; :::; = 0; ; :::; m Some importat developmets: ) Meyer (985), C wavelets. ) Mallat ad Meyer (987), Multiresolutio aalysis which gives a easy way to costruct wavelets ad also a fast algorithm (so called Pyramid algorithm, or Mallat algorithm). 3) Daubechies(988-99), Compactly supported C r wavelets (r is a positive umber), for example, Daubechies wavelets, Symlets, Coi ets. 4) Cohe, Daubechies ad Vial (993), Smooth wavelets o a iterval. 4 Good wavelets: vaishig momets The wavelet is said to have r vaishig momets if Z x (x) dx = 0 = 0; ; ; : : : ; r. Thus is orthogoal to all polyomials of degree r. For istace, r = for Haar wavelet. A fuctio f is said to be C ( > 0) o the iterval I R if there exists a costat C ad for every x I, there is a polyomial p x (y) of degree bc such that jf (x + y) p x (y)j C jyj, x + y I Lemma: If f is C o R ad has at least r = bc + vaishig momets, the f; j c C j(+=) Proof: Usig a chage of variable ad the vaishig momets property, Z f f; j = j= j v + j p j v (v) dv Z C j(+=) jv (v)j dv. 4
5 The scalig fuctio ' is said to have r vaishig momets if Z x ' (x) dx = 0, = ; ; : : : ; r. For = 0 recall that R x ' (x) dx =. Lemma: If f is C o R ad ' has at least r = bc + vaishig momets, the f; 'j j f j c ' C j(+=). Proof: Usig a chage of variable ad the vaishig momets property, f; ' j j f j Z f = j= j v + j f j p j v ' (v) dv Z C j(+=) jv ' (v)j dv. 5. Discrete wavelet trasformatio Multiresolutio aalysis (MRA) A MRA is a sequece of closed subspaces fv j ; j Zg i L (R) such that i)... V V V 0 V V :::; ii) f (x) V j () f (x) V j+ ; iii) [ jz V j = L (R) ; iv) there is a fuctio ' such that f' ( ) ; Zg is a orthoormal basis for V 0. There is a sequece fh g such that x p ' = h ' (x ) o ad for the sequece fg g = ( ) h we have ad More geerally JL j= where W j = spaf j = j= j See f i V J. The x p = g ' (x ) V j L Wj = V j+ W j = V J+! L (R), J!, ; Zg. f = J ' J = hf; ' J i ' J t f J ' J. 5
6 We see p j= ' j x = h j= ' j x i.e., (j )= ' j x = h j= ' j x the j ;0 = h j. or more geerally, j ;m = h j;+m. Similarly, (j )= j x = g j= ' j x the j ;0 = g j, or more geerally, j ;m = g j;+m. 6. Besov Balls The Besov sequece orms are de ed as follows. Let f = j0' j0 + j j j Suppose R ad 0 < p = q ad that we write s = + = =p. The b p;q = j0; lp + jsp j j j po =p ad geeral de itios for 0 < p; q is b p;q = j0; lp + jsq p q=p =q jjj. A cotrol of the Besov orm b p;q M is equivalet to js jj j p =p Mj or j j j M 0 j p due to the Berstei-type iequality below, or! () j j M 0 j p 6
7 Berstei-type Iequality Let j, K be a orthoormal sequece of fuctios satisfyig (i) P R j a j= (ii) max j a j=. The for all p, there exists costats C = a (a =a ) =p ad C = a (a =a ) =p such that for ay sequece = ( ; K) C j(= =p) p p j C j(= =p) p. 7. Nearly adaptive rate miimaxity We observe y i = i + z i ; i = ; ; :::; d; where z i ~N (0; ). De e the soft thresholdig estimator b as follows ^ i = where > 0. We have proved the followig result. Theorem P b p log d y i ; i = ; ; :::; d jy i j + ( + log d) +! d i ^. Questio: If M is a ellipsoid, ( ) (m; M) = f : a i i M, a = a + = () m ; i= i= ca you prove that P b p log C (log ) =(+)? I this lecture, we cosider Besov balls 8 8! 9 >< < q=p b p;q (M) = >: : ej0; lp = + j(+= =p)q j : j j p ; j =q 9 >= M >; If we apply b p log j to estimate ( j ) =;;:::; j i each resolutio, we show sup b p;q (M) P b C (log ) =(+) 7
8 or equivaletly sup P f b fbp;q (M) f C (log ) =(+). However, liear miimax rate is =(+) with = + = =p whe p < ad q. We will assume p + + = (or + =p = ( + )) for a techical reaso. Proof of early adaptive rate miimaxity Observe that ey j0 = e j0 + = z j0; j = j 0 ; = ; ; :::; j0 y j = j + = z j ; j = j 0 ; j 0 + ; :::; ; = ; ; :::; j where z j ~N (0; ) : I practice, j 0 = 4 or 5, ad we use y j0 to estimate j0; J satis es J =, ad we use 0 to estimate j for j J. Write ^ E = j0 + E bj j j 0j j0 + ( + log ) j ^ + j. Jj 0j jj Uder wea assumptios, it is true that j = o jj =(+) which meas eglectig y j for j J i practice is e. The it is eough to show j ^ C =(+). j 0j j 0j Let s start with p <, which is more excitig. De e j such that j = =(+), the we have j ^ = j ^ + j ^ j 0j<j =(+) + J>jj J>jj j ^. From the de itio of Besov balls, j p M j(+= =p), which implies Jjj j ^ p= j j j p j = O +p= p(+= =p) + = O +. 8
9 For p, the Jese s iequality gives j! =p j( =p) j j j p jj jj = O j( =p) j(+= =p) = O j, Remar. Liear miimaxity for p <? Let s cosider the case p = q <, the R L Bp;q (M) = R L B; (M) where = + = =p; ad R L B ; (M) =(+). For example, whe = p = q =, we have R L B ; (M) = but the optimal miimax rate is =3. Why? Recall that for p < i each resolutio j R L;j = if E (cy j j ) c R L = j P j j + P j Bp;q (M) t j0 + j 0j j P j j + P j Ad the max of P j uder the l p costrait j p M j(+= M j with = + = =p. =p) is 8. SURE Estimatio Dooho ad Johstoe (995, SURE). Setch of the proof. Cosider the sequece model where y i = i + z i, i = ; :::; d ad z i are idepedet ormal N(0; ) variables. Set r() = d P^. The stei s ubiased estimator of ris gives r(; L) = P (U ()) with P U () = d i= I (jyi j ) + y i I (jy i j ) + I (jy i j > ) = d # fi; jy i j g + P (jy i j ^ ) 9
10 Whe y(i) < y(i+), U () is a icreasig fuctio of. This implies Propositio: s = arg mi U () f0; jy j ; : : : ; jy jg. P sup ju () 0 p log d r (; L)j C log3= d d = : 9. Fuctioal Estimatio Quadratic fuctioal estimatio Model: Observe the sequece model: y i = i + = z i i:i:d: where z i N (0; ). The model comes from the white oise model (or may other models): dy (t) = f (t) dt + = db (t), t [0; ]. Let f i (t) ; i = ; ; : : :g be a orthoormal basis of L ([0; ]). The white oise problem is the equivalet to the sequece model with y i = h i ; yi ; i = h i ; fi ; ad z i = h i ; Bi i:i:d: N (0; ). Assumptio: Let = ( ; ; : : :). Assume ( ;M = : ) i i M. i= If the orthoormal basis is the Fourier basis, this assumptio correspods to the periodic Sobolev ball with smoothess. Now we may write the sequece model as P = fp ; ; ;M g where P ; is the joit distributio of idepedet y i N ( i ; =). Problem: Estimate Q = P i= i ( or R f with f = P i= i i ), ad determie the optimal miimax rate satisfyig lim if! bq sup P Q b Q : Solutio: Step : Achievig the optimal rate. De e m bq m = yi ; i= where m will be speci ed later. We ow P b Q m = 0 m i= i
11 ad var bqm = = = m i= m i= m i= = m var yi var z i p z i i var z i + 4 m i. i= + m i= var p z i i The ad Choose m such that i.e., 0 im+ i A M m 4 var bqm m + 4M. m = M m 4 M =(+4) m = =(+4). Thus we have a upper boud of the mea square error P bq Q M m 4 + 4M 8=(+4) +. So the optimal miimax covergece rate is = whe =4, or 8=(+4) whe < =4. Step : Ca ot do better. We tae P 0 = fp, = 0g ad P = fp ; sub g with sub = f : ji j = a; i m; i = 0; i m + g. where a ad m will be speci ed later. Simple calculatios give m i i m + a, i= m i i=! = m a 4, 4 i = ma 4. i=
12 Let m = =(4+) ad a = c (=) (+)=(4+). The m i i c ; i=! = c 4 (=) 8=(4+) ; 4 i = c 4 i i= i= Let c be a small costat, the sub ;M, ad the a ity P 0 P c for some positive costat c (why?). So (More details i class). sup P bq Q! i i= c (=) 8=(4+). 0. High dimesioal estimatio P 0 P
1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationLECTURE 11 LINEAR PROCESSES III: ASYMPTOTIC RESULTS
PRIL 7, 9 where LECTURE LINER PROCESSES III: SYMPTOTIC RESULTS (Phillips ad Solo (99) ad Phillips Lecture Notes o Statioary ad Nostatioary Time Series) I this lecture, we discuss the LLN ad CLT for a liear
More informationLecture 11 and 12: Basic estimation theory
Lecture ad 2: Basic estimatio theory Sprig 202 - EE 94 Networked estimatio ad cotrol Prof. Kha March 2 202 I. MAXIMUM-LIKELIHOOD ESTIMATORS The maximum likelihood priciple is deceptively simple. Louis
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationChapter 7 Isoperimetric problem
Chapter 7 Isoperimetric problem Recall that the isoperimetric problem (see the itroductio its coectio with ido s proble) is oe of the most classical problem of a shape optimizatio. It ca be formulated
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More information5 Birkhoff s Ergodic Theorem
5 Birkhoff s Ergodic Theorem Amog the most useful of the various geeralizatios of KolmogorovâĂŹs strog law of large umbers are the ergodic theorems of Birkhoff ad Kigma, which exted the validity of the
More informationMath Solutions to homework 6
Math 175 - Solutios to homework 6 Cédric De Groote November 16, 2017 Problem 1 (8.11 i the book): Let K be a compact Hermitia operator o a Hilbert space H ad let the kerel of K be {0}. Show that there
More informationEconomics 241B Relation to Method of Moments and Maximum Likelihood OLSE as a Maximum Likelihood Estimator
Ecoomics 24B Relatio to Method of Momets ad Maximum Likelihood OLSE as a Maximum Likelihood Estimator Uder Assumptio 5 we have speci ed the distributio of the error, so we ca estimate the model parameters
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationECE 901 Lecture 12: Complexity Regularization and the Squared Loss
ECE 90 Lecture : Complexity Regularizatio ad the Squared Loss R. Nowak 5/7/009 I the previous lectures we made use of the Cheroff/Hoeffdig bouds for our aalysis of classifier errors. Hoeffdig s iequality
More informationMa 4121: Introduction to Lebesgue Integration Solutions to Homework Assignment 5
Ma 42: Itroductio to Lebesgue Itegratio Solutios to Homework Assigmet 5 Prof. Wickerhauser Due Thursday, April th, 23 Please retur your solutios to the istructor by the ed of class o the due date. You
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationLecture 3. Properties of Summary Statistics: Sampling Distribution
Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationStat410 Probability and Statistics II (F16)
Some Basic Cocepts of Statistical Iferece (Sec 5.) Suppose we have a rv X that has a pdf/pmf deoted by f(x; θ) or p(x; θ), where θ is called the parameter. I previous lectures, we focus o probability problems
More informationRandom Matrices with Blocks of Intermediate Scale Strongly Correlated Band Matrices
Radom Matrices with Blocks of Itermediate Scale Strogly Correlated Bad Matrices Jiayi Tog Advisor: Dr. Todd Kemp May 30, 07 Departmet of Mathematics Uiversity of Califoria, Sa Diego Cotets Itroductio Notatio
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationEfficient GMM LECTURE 12 GMM II
DECEMBER 1 010 LECTURE 1 II Efficiet The estimator depeds o the choice of the weight matrix A. The efficiet estimator is the oe that has the smallest asymptotic variace amog all estimators defied by differet
More informationIt is often useful to approximate complicated functions using simpler ones. We consider the task of approximating a function by a polynomial.
Taylor Polyomials ad Taylor Series It is ofte useful to approximate complicated fuctios usig simpler oes We cosider the task of approximatig a fuctio by a polyomial If f is at least -times differetiable
More informationPAPER : IIT-JAM 2010
MATHEMATICS-MA (CODE A) Q.-Q.5: Oly oe optio is correct for each questio. Each questio carries (+6) marks for correct aswer ad ( ) marks for icorrect aswer.. Which of the followig coditios does NOT esure
More informationAbstract Vector Spaces. Abstract Vector Spaces
Astract Vector Spaces The process of astractio is critical i egieerig! Physical Device Data Storage Vector Space MRI machie Optical receiver 0 0 1 0 1 0 0 1 Icreasig astractio 6.1 Astract Vector Spaces
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationLecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationChapter 3 Inner Product Spaces. Hilbert Spaces
Chapter 3 Ier Product Spaces. Hilbert Spaces 3. Ier Product Spaces. Hilbert Spaces 3.- Defiitio. A ier product space is a vector space X with a ier product defied o X. A Hilbert space is a complete ier
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationIntroduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT
Itroductio to Extreme Value Theory Laures de Haa, ISM Japa, 202 Itroductio to Extreme Value Theory Laures de Haa Erasmus Uiversity Rotterdam, NL Uiversity of Lisbo, PT Itroductio to Extreme Value Theory
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationFilter banks. Separately, the lowpass and highpass filters are not invertible. removes the highest frequency 1/ 2and
Filter bas Separately, the lowpass ad highpass filters are ot ivertible T removes the highest frequecy / ad removes the lowest frequecy Together these filters separate the sigal ito low-frequecy ad high-frequecy
More informationMachine Learning Brett Bernstein
Machie Learig Brett Berstei Week 2 Lecture: Cocept Check Exercises Starred problems are optioal. Excess Risk Decompositio 1. Let X = Y = {1, 2,..., 10}, A = {1,..., 10, 11} ad suppose the data distributio
More informationTopics. Homework Problems. MATH 301 Introduction to Analysis Chapter Four Sequences. 1. Definition of convergence of sequences.
MATH 301 Itroductio to Aalysis Chapter Four Sequeces Topics 1. Defiitio of covergece of sequeces. 2. Fidig ad provig the limit of sequeces. 3. Bouded covergece theorem: Theorem 4.1.8. 4. Theorems 4.1.13
More informationProbability for mathematicians INDEPENDENCE TAU
Probability for mathematicias INDEPENDENCE TAU 2013 28 Cotets 3 Ifiite idepedet sequeces 28 3a Idepedet evets........................ 28 3b Idepedet radom variables.................. 33 3 Ifiite idepedet
More informationLecture 7: Density Estimation: k-nearest Neighbor and Basis Approach
STAT 425: Itroductio to Noparametric Statistics Witer 28 Lecture 7: Desity Estimatio: k-nearest Neighbor ad Basis Approach Istructor: Ye-Chi Che Referece: Sectio 8.4 of All of Noparametric Statistics.
More informationx x x Using a second Taylor polynomial with remainder, find the best constant C so that for x 0,
Math Activity 9( Due with Fial Eam) Usig first ad secod Taylor polyomials with remaider, show that for, 8 Usig a secod Taylor polyomial with remaider, fid the best costat C so that for, C 9 The th Derivative
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationMAS111 Convergence and Continuity
MAS Covergece ad Cotiuity Key Objectives At the ed of the course, studets should kow the followig topics ad be able to apply the basic priciples ad theorems therei to solvig various problems cocerig covergece
More information8.1 Introduction. 8. Nonparametric Inference Using Orthogonal Functions
8. Noparametric Iferece Usig Orthogoal Fuctios 1. Itroductio. Noparametric Regressio 3. Irregular Desigs 4. Desity Estimatio 5. Compariso of Methods 8.1 Itroductio Use a orthogoal basis to covert oparametric
More informationMINIMAX RATES OF CONVERGENCE AND OPTIMALITY OF BAYES FACTOR WAVELET REGRESSION ESTIMATORS UNDER POINTWISE RISKS
Statistica Siica 19 (2009) 1389 1406: Supplemet S1 S17 1 MINIMAX RATES OF ONVERGENE AND OPTIMALITY OF BAYES FATOR WAVELET REGRESSION ESTIMATORS UNDER POINTWISE RISKS Natalia Bochkia ad Theofais Sapatias
More informationJanuary 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
More informationMath 61CM - Solutions to homework 3
Math 6CM - Solutios to homework 3 Cédric De Groote October 2 th, 208 Problem : Let F be a field, m 0 a fixed oegative iteger ad let V = {a 0 + a x + + a m x m a 0,, a m F} be the vector space cosistig
More informationAssignment 5: Solutions
McGill Uiversity Departmet of Mathematics ad Statistics MATH 54 Aalysis, Fall 05 Assigmet 5: Solutios. Let y be a ubouded sequece of positive umbers satisfyig y + > y for all N. Let x be aother sequece
More informationIntroduction to Optimization Techniques
Itroductio to Optimizatio Techiques Basic Cocepts of Aalysis - Real Aalysis, Fuctioal Aalysis 1 Basic Cocepts of Aalysis Liear Vector Spaces Defiitio: A vector space X is a set of elemets called vectors
More informationINVERSE THEOREMS OF APPROXIMATION THEORY IN L p,α (R + )
Electroic Joural of Mathematical Aalysis ad Applicatios, Vol. 3(2) July 2015, pp. 92-99. ISSN: 2090-729(olie) http://fcag-egypt.com/jourals/ejmaa/ INVERSE THEOREMS OF APPROXIMATION THEORY IN L p,α (R +
More informationIntro to Learning Theory
Lecture 1, October 18, 2016 Itro to Learig Theory Ruth Urer 1 Machie Learig ad Learig Theory Comig soo 2 Formal Framework 21 Basic otios I our formal model for machie learig, the istaces to be classified
More informationAPPROXIMATION BY BERNSTEIN-CHLODOWSKY POLYNOMIALS
Hacettepe Joural of Mathematics ad Statistics Volume 32 (2003), 1 5 APPROXIMATION BY BERNSTEIN-CHLODOWSKY POLYNOMIALS E. İbili Received 27/06/2002 : Accepted 17/03/2003 Abstract The weighted approximatio
More information1 Lecture 2: Sequence, Series and power series (8/14/2012)
Summer Jump-Start Program for Aalysis, 202 Sog-Yig Li Lecture 2: Sequece, Series ad power series (8/4/202). More o sequeces Example.. Let {x } ad {y } be two bouded sequeces. Show lim sup (x + y ) lim
More informationLecture 19. sup y 1,..., yn B d n
STAT 06A: Polyomials of adom Variables Lecture date: Nov Lecture 19 Grothedieck s Iequality Scribe: Be Hough The scribes are based o a guest lecture by ya O Doell. I this lecture we prove Grothedieck s
More informationSolutions: Homework 3
Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe
More informationWe are mainly going to be concerned with power series in x, such as. (x)} converges - that is, lims N n
Review of Power Series, Power Series Solutios A power series i x - a is a ifiite series of the form c (x a) =c +c (x a)+(x a) +... We also call this a power series cetered at a. Ex. (x+) is cetered at
More informationUniversity of Colorado Denver Dept. Math. & Stat. Sciences Applied Analysis Preliminary Exam 13 January 2012, 10:00 am 2:00 pm. Good luck!
Uiversity of Colorado Dever Dept. Math. & Stat. Scieces Applied Aalysis Prelimiary Exam 13 Jauary 01, 10:00 am :00 pm Name: The proctor will let you read the followig coditios before the exam begis, ad
More information1 Approximating Integrals using Taylor Polynomials
Seughee Ye Ma 8: Week 7 Nov Week 7 Summary This week, we will lear how we ca approximate itegrals usig Taylor series ad umerical methods. Topics Page Approximatig Itegrals usig Taylor Polyomials. Defiitios................................................
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationThe variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
More informationON WELLPOSEDNESS QUADRATIC FUNCTION MINIMIZATION PROBLEM ON INTERSECTION OF TWO ELLIPSOIDS * M. JA]IMOVI], I. KRNI] 1.
Yugoslav Joural of Operatios Research 1 (00), Number 1, 49-60 ON WELLPOSEDNESS QUADRATIC FUNCTION MINIMIZATION PROBLEM ON INTERSECTION OF TWO ELLIPSOIDS M. JA]IMOVI], I. KRNI] Departmet of Mathematics
More informationLecture 6 Simple alternatives and the Neyman-Pearson lemma
STATS 00: Itroductio to Statistical Iferece Autum 06 Lecture 6 Simple alteratives ad the Neyma-Pearso lemma Last lecture, we discussed a umber of ways to costruct test statistics for testig a simple ull
More informationKernel density estimator
Jauary, 07 NONPARAMETRIC ERNEL DENSITY ESTIMATION I this lecture, we discuss kerel estimatio of probability desity fuctios PDF Noparametric desity estimatio is oe of the cetral problems i statistics I
More informationRiesz-Fischer Sequences and Lower Frame Bounds
Zeitschrift für Aalysis ud ihre Aweduge Joural for Aalysis ad its Applicatios Volume 1 (00), No., 305 314 Riesz-Fischer Sequeces ad Lower Frame Bouds P. Casazza, O. Christese, S. Li ad A. Lider Abstract.
More informationAn Introduction to Asymptotic Theory
A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu
More informationarxiv: v1 [math.pr] 13 Oct 2011
A tail iequality for quadratic forms of subgaussia radom vectors Daiel Hsu, Sham M. Kakade,, ad Tog Zhag 3 arxiv:0.84v math.pr] 3 Oct 0 Microsoft Research New Eglad Departmet of Statistics, Wharto School,
More informationA brief introduction to linear algebra
CHAPTER 6 A brief itroductio to liear algebra 1. Vector spaces ad liear maps I what follows, fix K 2{Q, R, C}. More geerally, K ca be ay field. 1.1. Vector spaces. Motivated by our ituitio of addig ad
More informationOn the convergence rates of Gladyshev s Hurst index estimator
Noliear Aalysis: Modellig ad Cotrol, 2010, Vol 15, No 4, 445 450 O the covergece rates of Gladyshev s Hurst idex estimator K Kubilius 1, D Melichov 2 1 Istitute of Mathematics ad Iformatics, Vilius Uiversity
More informationHOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
More informationLecture 27: Optimal Estimators and Functional Delta Method
Stat210B: Theoretical Statistics Lecture Date: April 19, 2007 Lecture 27: Optimal Estimators ad Fuctioal Delta Method Lecturer: Michael I. Jorda Scribe: Guilherme V. Rocha 1 Achievig Optimal Estimators
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationSOME THEORY AND PRACTICE OF STATISTICS by Howard G. Tucker
SOME THEORY AND PRACTICE OF STATISTICS by Howard G. Tucker CHAPTER 9. POINT ESTIMATION 9. Covergece i Probability. The bases of poit estimatio have already bee laid out i previous chapters. I chapter 5
More informationFirst Year Quantitative Comp Exam Spring, Part I - 203A. f X (x) = 0 otherwise
First Year Quatitative Comp Exam Sprig, 2012 Istructio: There are three parts. Aswer every questio i every part. Questio I-1 Part I - 203A A radom variable X is distributed with the margial desity: >
More informationECE 901 Lecture 13: Maximum Likelihood Estimation
ECE 90 Lecture 3: Maximum Likelihood Estimatio R. Nowak 5/7/009 The focus of this lecture is to cosider aother approach to learig based o maximum likelihood estimatio. Ulike earlier approaches cosidered
More informationMATH301 Real Analysis (2008 Fall) Tutorial Note #7. k=1 f k (x) converges pointwise to S(x) on E if and
MATH01 Real Aalysis (2008 Fall) Tutorial Note #7 Sequece ad Series of fuctio 1: Poitwise Covergece ad Uiform Covergece Part I: Poitwise Covergece Defiitio of poitwise covergece: A sequece of fuctios f
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationSelf-normalized deviation inequalities with application to t-statistic
Self-ormalized deviatio iequalities with applicatio to t-statistic Xiequa Fa Ceter for Applied Mathematics, Tiaji Uiversity, 30007 Tiaji, Chia Abstract Let ξ i i 1 be a sequece of idepedet ad symmetric
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationSTAT Homework 2 - Solutions
STAT-36700 Homework - Solutios Fall 08 September 4, 08 This cotais solutios for Homework. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better isight.
More informationLecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound
Lecture 7 Ageda for the lecture Gaussia chael with average power costraits Capacity of additive Gaussia oise chael ad the sphere packig boud 7. Additive Gaussia oise chael Up to this poit, we have bee
More informatione to approximate (using 4
Review: Taylor Polyomials ad Power Series Fid the iterval of covergece for the series Fid a series for f ( ) d ad fid its iterval of covergece Let f( ) Let f arcta a) Fid the rd degree Maclauri polyomial
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationLecture 10 October Minimaxity and least favorable prior sequences
STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least
More informationMath 220B Final Exam Solutions March 18, 2002
Math 0B Fial Exam Solutios March 18, 00 1. (1 poits) (a) (6 poits) Fid the Gree s fuctio for the tilted half-plae {(x 1, x ) R : x 1 + x > 0}. For x (x 1, x ), y (y 1, y ), express your Gree s fuctio G(x,
More informationMi-Hwa Ko and Tae-Sung Kim
J. Korea Math. Soc. 42 2005), No. 5, pp. 949 957 ALMOST SURE CONVERGENCE FOR WEIGHTED SUMS OF NEGATIVELY ORTHANT DEPENDENT RANDOM VARIABLES Mi-Hwa Ko ad Tae-Sug Kim Abstract. For weighted sum of a sequece
More informationSection 11.8: Power Series
Sectio 11.8: Power Series 1. Power Series I this sectio, we cosider geeralizig the cocept of a series. Recall that a series is a ifiite sum of umbers a. We ca talk about whether or ot it coverges ad i
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationf n (x) f m (x) < ɛ/3 for all x A. By continuity of f n and f m we can find δ > 0 such that d(x, x 0 ) < δ implies that
Lecture 15 We have see that a sequece of cotiuous fuctios which is uiformly coverget produces a limit fuctio which is also cotiuous. We shall stregthe this result ow. Theorem 1 Let f : X R or (C) be a
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationMath 2784 (or 2794W) University of Connecticut
ORDERS OF GROWTH PAT SMITH Math 2784 (or 2794W) Uiversity of Coecticut Date: Mar. 2, 22. ORDERS OF GROWTH. Itroductio Gaiig a ituitive feel for the relative growth of fuctios is importat if you really
More informationSolutions to home assignments (sketches)
Matematiska Istitutioe Peter Kumli 26th May 2004 TMA401 Fuctioal Aalysis MAN670 Applied Fuctioal Aalysis 4th quarter 2003/2004 All documet cocerig the course ca be foud o the course home page: http://www.math.chalmers.se/math/grudutb/cth/tma401/
More informationIf a subset E of R contains no open interval, is it of zero measure? For instance, is the set of irrationals in [0, 1] is of measure zero?
2 Lebesgue Measure I Chapter 1 we defied the cocept of a set of measure zero, ad we have observed that every coutable set is of measure zero. Here are some atural questios: If a subset E of R cotais a
More informationLearning Theory: Lecture Notes
Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic
More informationMachine Learning Brett Bernstein
Machie Learig Brett Berstei Week Lecture: Cocept Check Exercises Starred problems are optioal. Statistical Learig Theory. Suppose A = Y = R ad X is some other set. Furthermore, assume P X Y is a discrete
More informationSupplemental Material: Proofs
Proof to Theorem Supplemetal Material: Proofs Proof. Let be the miimal umber of traiig items to esure a uique solutio θ. First cosider the case. It happes if ad oly if θ ad Rak(A) d, which is a special
More informationReview Problems 1. ICME and MS&E Refresher Course September 19, 2011 B = C = AB = A = A 2 = A 3... C 2 = C 3 = =
Review Problems ICME ad MS&E Refresher Course September 9, 0 Warm-up problems. For the followig matrices A = 0 B = C = AB = 0 fid all powers A,A 3,(which is A times A),... ad B,B 3,... ad C,C 3,... Solutio:
More informationApply change-of-basis formula to rewrite x as a linear combination of eigenvectors v j.
Eigevalue-Eigevector Istructor: Nam Su Wag eigemcd Ay vector i real Euclidea space of dimesio ca be uiquely epressed as a liear combiatio of liearly idepedet vectors (ie, basis) g j, j,,, α g α g α g α
More information2.4.2 A Theorem About Absolutely Convergent Series
0 Versio of August 27, 200 CHAPTER 2. INFINITE SERIES Add these two series: + 3 2 + 5 + 7 4 + 9 + 6 +... = 3 l 2. (2.20) 2 Sice the reciprocal of each iteger occurs exactly oce i the last series, we would
More informationMAT1026 Calculus II Basic Convergence Tests for Series
MAT026 Calculus II Basic Covergece Tests for Series Egi MERMUT 202.03.08 Dokuz Eylül Uiversity Faculty of Sciece Departmet of Mathematics İzmir/TURKEY Cotets Mootoe Covergece Theorem 2 2 Series of Real
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationLecture 13: Maximum Likelihood Estimation
ECE90 Sprig 007 Statistical Learig Theory Istructor: R. Nowak Lecture 3: Maximum Likelihood Estimatio Summary of Lecture I the last lecture we derived a risk (MSE) boud for regressio problems; i.e., select
More information