MATH 829: Introduction to Data Mining and Analysis Consistency of Linear Regression
|
|
|
- Emery Phillips
- 5 years ago
- Views:
Transcription
1 1/9 MATH 829: Introduction to Data Mining and Analysis Consistency of Linear Regression Dominique Guillot Deartments of Mathematical Sciences University of Delaware February 15, 2016
2 Distribution of regression coecients 2/9 Observations Y = (y i ) R n, X = (x ij ) R n.
3 Distribution of regression coecients 2/9 Observations Y = (y i ) R n, X = (x ij ) R n. Assumtions: 1 Y i = β 1 X i,1 + + β X i, + ɛ i (ɛ i = error).
4 2/9 Distribution of regression coecients Observations Y = (y i ) R n, X = (x ij ) R n. Assumtions: 1 Y i = β 1 X i,1 + + β X i, + ɛ i (ɛ i = error). In other words: Y = Xβ + ɛ. (β = (β 1,... β ) is a xed unknown vector)
5 2/9 Distribution of regression coecients Observations Y = (y i ) R n, X = (x ij ) R n. Assumtions: 1 Y i = β 1 X i,1 + + β X i, + ɛ i (ɛ i = error). In other words: Y = Xβ + ɛ. (β = (β 1,... β ) is a xed unknown vector) 2 x ij are non-random. ɛ i are random.
6 2/9 Distribution of regression coecients Observations Y = (y i ) R n, X = (x ij ) R n. Assumtions: 1 Y i = β 1 X i,1 + + β X i, + ɛ i (ɛ i = error). In other words: Y = Xβ + ɛ. (β = (β 1,... β ) is a xed unknown vector) 2 x ij are non-random. ɛ i are random. 3 ɛ i are indeendent N(0, σ 2 ). We have ˆβ = (X T X) 1 X T Y.
7 2/9 Distribution of regression coecients Observations Y = (y i ) R n, X = (x ij ) R n. Assumtions: 1 Y i = β 1 X i,1 + + β X i, + ɛ i (ɛ i = error). In other words: Y = Xβ + ɛ. (β = (β 1,... β ) is a xed unknown vector) 2 x ij are non-random. ɛ i are random. 3 ɛ i are indeendent N(0, σ 2 ). We have ˆβ = (X T X) 1 X T Y. What is the distribution of ˆβ?
8 Multivariate normal distribution 3/9 Recall: X = (X 1,..., X ) N(µ, Σ) where µ R, Σ = (σ ij ) R is ositive denite, if 1 P (X A) = e 1 (2π) 2 (x µ)t Σ 1 (x µ) dx 1... dx. det Σ A
9 Multivariate normal distribution 3/9 Recall: X = (X 1,..., X ) N(µ, Σ) where µ R, Σ = (σ ij ) R is ositive denite, if 1 P (X A) = e 1 (2π) 2 (x µ)t Σ 1 (x µ) dx 1... dx. det Σ A Bivariate case:
10 Multivariate normal distribution 3/9 Recall: X = (X 1,..., X ) N(µ, Σ) where µ R, Σ = (σ ij ) R is ositive denite, if 1 P (X A) = e 1 (2π) 2 (x µ)t Σ 1 (x µ) dx 1... dx. det Σ A Bivariate case: We have E(X) = µ, Cov(X i, X j ) = σ ij.
11 Multivariate normal distribution 3/9 Recall: X = (X 1,..., X ) N(µ, Σ) where µ R, Σ = (σ ij ) R is ositive denite, if 1 P (X A) = e 1 (2π) 2 (x µ)t Σ 1 (x µ) dx 1... dx. det Σ A Bivariate case: We have E(X) = µ, Cov(X i, X j ) = σ ij. If Y = c + BX, where c R and B R m, then Y N(c + Bµ, BΣB T ).
12 Distribution of the regression coecients (cont.) 4/9 Back to our roblem: Y = Xβ + ɛ where ɛ i are iid N(0, σ 2 ). We have Y N(Xβ, σ 2 I).
13 Distribution of the regression coecients (cont.) 4/9 Back to our roblem: Y = Xβ + ɛ where ɛ i are iid N(0, σ 2 ). We have Y N(Xβ, σ 2 I). Therefore, ˆβ = (X T X) 1 X T Y N(β, σ 2 (X T X) 1 ).
14 Distribution of the regression coecients (cont.) 4/9 Back to our roblem: Y = Xβ + ɛ where ɛ i are iid N(0, σ 2 ). We have Y N(Xβ, σ 2 I). Therefore, ˆβ = (X T X) 1 X T Y N(β, σ 2 (X T X) 1 ). In articular, Thus, ˆβ is unbiased. E( ˆβ) = β.
15 Statistical consistency of least squares 5/9 We saw that E( ˆβ) = β.
16 Statistical consistency of least squares 5/9 We saw that E( ˆβ) = β. What haens as the samle size n goes to innity? We exect ˆβ = ˆβ(n) β.
17 Statistical consistency of least squares 5/9 We saw that E( ˆβ) = β. What haens as the samle size n goes to innity? We exect ˆβ = ˆβ(n) β. A sequence of estimators {θ n } n=1 of a arameter θ is said to be consistent if θ n θ in robability (θ n θ) as n.
18 Statistical consistency of least squares 5/9 We saw that E( ˆβ) = β. What haens as the samle size n goes to innity? We exect ˆβ = ˆβ(n) β. A sequence of estimators {θ n } n=1 of a arameter θ is said to be consistent if θ n θ in robability (θ n θ) as n. (Recall: θ n θ if for every ɛ > 0, lim P ( θ n θ ɛ) = 0. n
19 Statistical consistency of least squares 5/9 We saw that E( ˆβ) = β. What haens as the samle size n goes to innity? We exect ˆβ = ˆβ(n) β. A sequence of estimators {θ n } n=1 of a arameter θ is said to be consistent if θ n θ in robability (θ n θ) as n. (Recall: θ n θ if for every ɛ > 0, lim P ( θ n θ ɛ) = 0. n In order to rove that ˆβ n (estimator with n samles) is consistent, we will make some assumtions on the data generating model.
20 Statistical consistency of least squares 5/9 We saw that E( ˆβ) = β. What haens as the samle size n goes to innity? We exect ˆβ = ˆβ(n) β. A sequence of estimators {θ n } n=1 of a arameter θ is said to be consistent if θ n θ in robability (θ n θ) as n. (Recall: θ n θ if for every ɛ > 0, lim P ( θ n θ ɛ) = 0. n In order to rove that ˆβ n (estimator with n samles) is consistent, we will make some assumtions on the data generating model. (Without any assumtions, nothing revents the observations to be all the same for examle... )
21 Statistical consistency of least squares (cont.) 6/9 Observations: y = (y i ) R n, X = (x ij ) R n.
22 Statistical consistency of least squares (cont.) 6/9 Observations: y = (y i ) R n, X = (x ij ) R n. Let x i := (x i,1,..., x i,n ) R (i = 1,..., n).
23 Statistical consistency of least squares (cont.) 6/9 Observations: y = (y i ) R n, X = (x ij ) R n. Let x i := (x i,1,..., x i,n ) R (i = 1,..., n). We will assume: 1 (x i ) n i=1 are iid random vectors. 2 y i = β 1 x i,1 + + β x i, + ɛ i where ɛ i are iid N(0, σ 2 ). 3 The error ɛ i is indeendent of x i. 4 Ex 2 ij < (nite second moment). 5 Q = E(x i x T i ) R is invertible.
24 Statistical consistency of least squares (cont.) 6/9 Observations: y = (y i ) R n, X = (x ij ) R n. Let x i := (x i,1,..., x i,n ) R (i = 1,..., n). We will assume: 1 (x i ) n i=1 are iid random vectors. 2 y i = β 1 x i,1 + + β x i, + ɛ i where ɛ i are iid N(0, σ 2 ). 3 The error ɛ i is indeendent of x i. 4 Ex 2 ij < (nite second moment). 5 Q = E(x i x T i ) R is invertible. Under these assumtions, we have the following theorem. Theorem: Let ˆβ n = (X T X) 1 X T y. Then, under the above assumtions, we have ˆβ n β.
25 Background for the roof 7/9 Recall: Weak law of large numbers: Let (X i ) i=1 be iid random variables with nite rst moment E( X i ) <. Let µ := E(X i ). Then X n := 1 n X i µ. n i=1
26 Background for the roof 7/9 Recall: Weak law of large numbers: Let (X i ) i=1 be iid random variables with nite rst moment E( X i ) <. Let µ := E(X i ). Then X n := 1 n X i µ. n i=1 Continuous maing theorem: Let S, S be metric saces. Suose (X i ) i=1 are S-valued random variables such that X i X. Let g : S S. Denote by D g the set of oints in S where g is discontinuous and suose P (X D g ) = 0. Then g(x n ) g(x).
27 Proof of the theorem 8/9 We have ˆβ = (X T X) 1 X T y = ( 1 n n x i x T i i=1 ) 1 ( 1 n ) n x i y i. i=1
28 Proof of the theorem 8/9 We have ˆβ = (X T X) 1 X T y = Using CauchySchwarz, ( 1 n n x i x T i i=1 ) 1 ( 1 n ) n x i y i. i=1 E( x ij x ik ) (E(x 2 ij)e(x 2 ik ))1/2 <.
29 Proof of the theorem 8/9 We have ˆβ = (X T X) 1 X T y = Using CauchySchwarz, ( 1 n n x i x T i i=1 ) 1 ( 1 n ) n x i y i. i=1 E( x ij x ik ) (E(x 2 ij)e(x 2 ik ))1/2 <. In a similar way, we rove that E( x ij y i ) <.
30 Proof of the theorem 8/9 We have ˆβ = (X T X) 1 X T y = Using CauchySchwarz, ( 1 n n x i x T i i=1 ) 1 ( 1 n ) n x i y i. i=1 E( x ij x ik ) (E(x 2 ij)e(x 2 ik ))1/2 <. In a similar way, we rove that E( x ij y i ) <. By the weak law of large numbers, we obtain 1 n 1 n n x i x T i i=1 E(x i x T i ) = Q, n x i y i E(xi y i ). i=1
31 Proof of the theorem (cont.) 9/9 Using the continuous maing theorem, we obtain ˆβ n E(xi x T i ) 1 E(x i y i ). (dene g : R R R by g(a, b) = A 1 b.)
32 Proof of the theorem (cont.) 9/9 Using the continuous maing theorem, we obtain ˆβ n E(xi x T i ) 1 E(x i y i ). (dene g : R R R by g(a, b) = A 1 b.) Recall: y i = x T i β + ɛ i. So x i y i = x i x T i β + x i ɛ i.
33 Proof of the theorem (cont.) 9/9 Using the continuous maing theorem, we obtain ˆβ n E(xi x T i ) 1 E(x i y i ). (dene g : R R R by g(a, b) = A 1 b.) Recall: y i = x T i β + ɛ i. So x i y i = x i x T i β + x i ɛ i. Taking exectations, E(x i y i ) = E(x i x T i )β + E(x i ɛ i ).
34 Proof of the theorem (cont.) 9/9 Using the continuous maing theorem, we obtain ˆβ n E(xi x T i ) 1 E(x i y i ). (dene g : R R R by g(a, b) = A 1 b.) Recall: y i = x T i β + ɛ i. So Taking exectations, x i y i = x i x T i β + x i ɛ i. E(x i y i ) = E(x i x T i )β + E(x i ɛ i ). Note that E(x i ɛ i ) = 0 since x i and ɛ i are indeendent by assumtion.
35 Proof of the theorem (cont.) and so ˆβ n β. 9/9 Using the continuous maing theorem, we obtain ˆβ n E(xi x T i ) 1 E(x i y i ). (dene g : R R R by g(a, b) = A 1 b.) Recall: y i = x T i β + ɛ i. So Taking exectations, x i y i = x i x T i β + x i ɛ i. E(x i y i ) = E(x i x T i )β + E(x i ɛ i ). Note that E(x i ɛ i ) = 0 since x i and ɛ i are indeendent by assumtion. We conclude that β = E(x i x T i ) 1 E(x i y i )
MATH 829: Introduction to Data Mining and Analysis Graphical Models II - Gaussian Graphical Models
1/13 MATH 829: Introduction to Data Mining and Analysis Graphical Models II - Gaussian Graphical Models Dominique Guillot Departments of Mathematical Sciences University of Delaware May 4, 2016 Recall
MATH 829: Introduction to Data Mining and Analysis Linear Regression: statistical tests
1/16 MATH 829: Introduction to Data Mining and Analysis Linear Regression: statistical tests Dominique Guillot Departments of Mathematical Sciences University of Delaware February 17, 2016 Statistical
MATH 829: Introduction to Data Mining and Analysis Least angle regression
1/14 MATH 829: Introduction to Data Mining and Analysis Least angle regression Dominique Guillot Departments of Mathematical Sciences University of Delaware February 29, 2016 Least angle regression (LARS)
Moment Generating Function. STAT/MTHE 353: 5 Moment Generating Functions and Multivariate Normal Distribution
Moment Generating Function STAT/MTHE 353: 5 Moment Generating Functions and Multivariate Normal Distribution T. Linder Queen s University Winter 07 Definition Let X (X,...,X n ) T be a random vector and
Review of Probability Theory II
Review of Probability Theory II January 9-3, 008 Exectation If the samle sace Ω = {ω, ω,...} is countable and g is a real-valued function, then we define the exected value or the exectation of a function
Lecture 2: Consistency of M-estimators
Lecture 2: Instructor: Deartment of Economics Stanford University Preared by Wenbo Zhou, Renmin University References Takeshi Amemiya, 1985, Advanced Econometrics, Harvard University Press Newey and McFadden,
Chapter 7: Special Distributions
This chater first resents some imortant distributions, and then develos the largesamle distribution theory which is crucial in estimation and statistical inference Discrete distributions The Bernoulli
Elements of Asymptotic Theory. James L. Powell Department of Economics University of California, Berkeley
Elements of Asymtotic Theory James L. Powell Deartment of Economics University of California, Berkeley Objectives of Asymtotic Theory While exact results are available for, say, the distribution of the
1 Extremum Estimators
FINC 9311-21 Financial Econometrics Handout Jialin Yu 1 Extremum Estimators Let θ 0 be a vector of k 1 unknown arameters. Extremum estimators: estimators obtained by maximizing or minimizing some objective
B8.1 Martingales Through Measure Theory. Concept of independence
B8.1 Martingales Through Measure Theory Concet of indeendence Motivated by the notion of indeendent events in relims robability, we have generalized the concet of indeendence to families of σ-algebras.
MATH 829: Introduction to Data Mining and Analysis Graphical Models I
MATH 829: Introduction to Data Mining and Analysis Graphical Models I Dominique Guillot Departments of Mathematical Sciences University of Delaware May 2, 2016 1/12 Independence and conditional independence:
Elements of Asymptotic Theory. James L. Powell Department of Economics University of California, Berkeley
Elements of Asymtotic Theory James L. Powell Deartment of Economics University of California, Berkeley Objectives of Asymtotic Theory While exact results are available for, say, the distribution of the
MATH 829: Introduction to Data Mining and Analysis Support vector machines and kernels
1/12 MATH 829: Introduction to Data Mining and Analysis Support vector machines and kernels Dominique Guillot Departments of Mathematical Sciences University of Delaware March 14, 2016 Separating sets:
Tsung-Lin Cheng and Yuan-Shih Chow. Taipei115,Taiwan R.O.C.
A Generalization and Alication of McLeish's Central Limit Theorem by Tsung-Lin Cheng and Yuan-Shih Chow Institute of Statistical Science Academia Sinica Taiei5,Taiwan R.O.C. hcho3@stat.sinica.edu.tw Abstract.
Maximum Likelihood Asymptotic Theory. Eduardo Rossi University of Pavia
Maximum Likelihood Asymtotic Theory Eduardo Rossi University of Pavia Slutsky s Theorem, Cramer s Theorem Slutsky s Theorem Let {X N } be a random sequence converging in robability to a constant a, and
Notes on Random Vectors and Multivariate Normal
MATH 590 Spring 06 Notes on Random Vectors and Multivariate Normal Properties of Random Vectors If X,, X n are random variables, then X = X,, X n ) is a random vector, with the cumulative distribution
Probability- the good parts version. I. Random variables and their distributions; continuous random variables.
Probability- the good arts version I. Random variables and their distributions; continuous random variables. A random variable (r.v) X is continuous if its distribution is given by a robability density
San Francisco State University ECON 851 Summer Problem Set 1
San Francisco State University Michael Bar ECON 85 Summer 05 Problem Set. Suose that the wea reference relation % on L is transitive. Prove that the strict reference relation is transitive. Let A; B; C
Asymptotic theory for linear regression and IV estimation
Asymtotic theory for linear regression and IV estimation Jean-Marie Dufour McGill University First version: November 20 Revised: December 20 his version: December 20 Comiled: December 3, 20, : his work
Lecture 6. 2 Recurrence/transience, harmonic functions and martingales
Lecture 6 Classification of states We have shown that all states of an irreducible countable state Markov chain must of the same tye. This gives rise to the following classification. Definition. [Classification
LEIBNIZ SEMINORMS IN PROBABILITY SPACES
LEIBNIZ SEMINORMS IN PROBABILITY SPACES ÁDÁM BESENYEI AND ZOLTÁN LÉKA Abstract. In this aer we study the (strong) Leibniz roerty of centered moments of bounded random variables. We shall answer a question
Real Analysis 1 Fall Homework 3. a n.
eal Analysis Fall 06 Homework 3. Let and consider the measure sace N, P, µ, where µ is counting measure. That is, if N, then µ equals the number of elements in if is finite; µ = otherwise. One usually
Outline for today. Maximum likelihood estimation. Computation with multivariate normal distributions. Multivariate normal distribution
Outline for today Maximum likelihood estimation Rasmus Waageetersen Deartment of Mathematics Aalborg University Denmark October 30, 2007 the multivariate normal distribution linear and linear mixed models
LECTURE 7 NOTES. x n. d x if. E [g(x n )] E [g(x)]
![LECTURE 7 NOTES. x n. d x if. E [g(x n )] E [g(x)]](/thumbs/82/85236525.jpg "LECTURE 7 NOTES. x n. d x if. E [g(x n )] E [g(x)]")
LECTURE 7 NOTES 1. Convergence of random variables. Before delving into the large samle roerties of the MLE, we review some concets from large samle theory. 1. Convergence in robability: x n x if, for
Use of Transformations and the Repeated Statement in PROC GLM in SAS Ed Stanek
Use of Transformations and the Reeated Statement in PROC GLM in SAS Ed Stanek Introduction We describe how the Reeated Statement in PROC GLM in SAS transforms the data to rovide tests of hyotheses of interest.
On Z p -norms of random vectors
On Z -norms of random vectors Rafa l Lata la Abstract To any n-dimensional random vector X we may associate its L -centroid body Z X and the corresonding norm. We formulate a conjecture concerning the
ECON 4130 Supplementary Exercises 1-4
HG Set. 0 ECON 430 Sulementary Exercises - 4 Exercise Quantiles (ercentiles). Let X be a continuous random variable (rv.) with df f( x ) and cdf F( x ). For 0< < we define -th quantile (or 00-th ercentile),
Principal Components Analysis and Unsupervised Hebbian Learning
Princial Comonents Analysis and Unsuervised Hebbian Learning Robert Jacobs Deartment of Brain & Cognitive Sciences University of Rochester Rochester, NY 1467, USA August 8, 008 Reference: Much of the material
Notes on Instrumental Variables Methods
Notes on Instrumental Variables Methods Michele Pellizzari IGIER-Bocconi, IZA and frdb 1 The Instrumental Variable Estimator Instrumental variable estimation is the classical solution to the roblem of
MATH 567: Mathematical Techniques in Data Science Linear Regression: old and new
1/14 MATH 567: Mathematical Techniques in Data Science Linear Regression: old and new Dominique Guillot Departments of Mathematical Sciences University of Delaware February 13, 2017 Linear Regression:
Mollifiers and its applications in L p (Ω) space
Mollifiers and its alications in L () sace MA Shiqi Deartment of Mathematics, Hong Kong Batist University November 19, 2016 Abstract This note gives definition of mollifier and mollification. We illustrate
Lecture 23 Maximum Likelihood Estimation and Bayesian Inference
Lecture 23 Maximum Likelihood Estimation and Bayesian Inference Thais Paiva STA 111 - Summer 2013 Term II August 7, 2013 1 / 31 Thais Paiva STA 111 - Summer 2013 Term II Lecture 23, 08/07/2013 Lecture
On the minimax inequality and its application to existence of three solutions for elliptic equations with Dirichlet boundary condition
ISSN 1 746-7233 England UK World Journal of Modelling and Simulation Vol. 3 (2007) No. 2. 83-89 On the minimax inequality and its alication to existence of three solutions for ellitic equations with Dirichlet
Regression and Statistical Inference
Regression and Statistical Inference Walid Mnif wmnif@uwo.ca Department of Applied Mathematics The University of Western Ontario, London, Canada 1 Elements of Probability 2 Elements of Probability CDF&PDF
Analysis of some entrance probabilities for killed birth-death processes
Analysis of some entrance robabilities for killed birth-death rocesses Master s Thesis O.J.G. van der Velde Suervisor: Dr. F.M. Sieksma July 5, 207 Mathematical Institute, Leiden University Contents Introduction
STAT Homework 8 - Solutions
STAT-36700 Homework 8 - Solutions Fall 208 November 3, 208 This contains solutions for Homework 4. lease note that we have included several additional comments and approaches to the problems to give you
1 Probability Spaces and Random Variables
1 Probability Saces and Random Variables 1.1 Probability saces Ω: samle sace consisting of elementary events (or samle oints). F : the set of events P: robability 1.2 Kolmogorov s axioms Definition 1.2.1
1 Gambler s Ruin Problem
Coyright c 2017 by Karl Sigman 1 Gambler s Ruin Problem Let N 2 be an integer and let 1 i N 1. Consider a gambler who starts with an initial fortune of $i and then on each successive gamble either wins
MATH 829: Introduction to Data Mining and Analysis Computing the lasso solution
1/16 MATH 829: Introduction to Data Mining and Analysis Computing the lasso solution Dominique Guillot Departments of Mathematical Sciences University of Delaware February 26, 2016 Computing the lasso
6 Stationary Distributions
6 Stationary Distributions 6. Definition and Examles Definition 6.. Let {X n } be a Markov chain on S with transition robability matrix P. A distribution π on S is called stationary (or invariant) if π
General Random Variables
Chater General Random Variables. Law of a Random Variable Thus far we have considered onl random variables whose domain and range are discrete. We now consider a general random variable X! defined on the
1 1 c (a) 1 (b) 1 Figure 1: (a) First ath followed by salesman in the stris method. (b) Alternative ath. 4. D = distance travelled closing the loo. Th
18.415/6.854 Advanced Algorithms ovember 7, 1996 Euclidean TSP (art I) Lecturer: Michel X. Goemans MIT These notes are based on scribe notes by Marios Paaefthymiou and Mike Klugerman. 1 Euclidean TSP Consider
ECON Answers Homework #2
ECON 33 - Answers Homework #2 Exercise : Denote by x the number of containers of tye H roduced, y the number of containers of tye T and z the number of containers of tye I. There are 3 inut equations that
Existence and nonexistence of positive solutions for quasilinear elliptic systems
ISSN 1746-7233, England, UK World Journal of Modelling and Simulation Vol. 4 (2008) No. 1,. 44-48 Existence and nonexistence of ositive solutions for uasilinear ellitic systems G. A. Afrouzi, H. Ghorbani
Lecture 3 Consistency of Extremum Estimators 1
Lecture 3 Consistency of Extremum Estimators 1 This lecture shows how one can obtain consistency of extremum estimators. It also shows how one can find the robability limit of extremum estimators in cases
Elementary Analysis in Q p
Elementary Analysis in Q Hannah Hutter, May Szedlák, Phili Wirth November 17, 2011 This reort follows very closely the book of Svetlana Katok 1. 1 Sequences and Series In this section we will see some
Matrix Approach to Simple Linear Regression: An Overview
Matrix Approach to Simple Linear Regression: An Overview Aspects of matrices that you should know: Definition of a matrix Addition/subtraction/multiplication of matrices Symmetric/diagonal/identity matrix
Computing the covariance of two Brownian area integrals
Statistica Neerlandica () Vol. 56, nr.,. ±9 Comuting the covariance of two Brownian area integrals J. A. Wellner* University of Washington, Statistics, Box 3543, Seattle, Washington 9895-43, U.S.A. R.
Asymptotically Optimal Simulation Allocation under Dependent Sampling
Asymtotically Otimal Simulation Allocation under Deendent Samling Xiaoing Xiong The Robert H. Smith School of Business, University of Maryland, College Park, MD 20742-1815, USA, xiaoingx@yahoo.com Sandee
LIST OF FORMULAS FOR STK1100 AND STK1110
LIST OF FORMULAS FOR STK1100 AND STK1110 (Version of 11. November 2015) 1. Probability Let A, B, A 1, A 2,..., B 1, B 2,... be events, that is, subsets of a sample space Ω. a) Axioms: A probability function
7. Introduction to Large Sample Theory
7. Introuction to Large Samle Theory Hayashi. 88-97/109-133 Avance Econometrics I, Autumn 2010, Large-Samle Theory 1 Introuction We looke at finite-samle roerties of the OLS estimator an its associate
MATH 829: Introduction to Data Mining and Analysis Linear Regression: old and new
1/15 MATH 829: Introduction to Data Mining and Analysis Linear Regression: old and new Dominique Guillot Departments of Mathematical Sciences University of Delaware February 10, 2016 Linear Regression:
Probability Lecture III (August, 2006)
robability Lecture III (August, 2006) 1 Some roperties of Random Vectors and Matrices We generalize univariate notions in this section. Definition 1 Let U = U ij k l, a matrix of random variables. Suppose
YOUNESS LAMZOURI H 2. The purpose of this note is to improve the error term in this asymptotic formula. H 2 (log log H) 3 ζ(3) H2 + O
ON THE AVERAGE OF THE NUMBER OF IMAGINARY QUADRATIC FIELDS WITH A GIVEN CLASS NUMBER YOUNESS LAMZOURI Abstract Let Fh be the number of imaginary quadratic fields with class number h In this note we imrove
Estimating Time-Series Models
Estimating ime-series Models he Box-Jenkins methodology for tting a model to a scalar time series fx t g consists of ve stes:. Decide on the order of di erencing d that is needed to roduce a stationary
Lecture: Condorcet s Theorem
Social Networs and Social Choice Lecture Date: August 3, 00 Lecture: Condorcet s Theorem Lecturer: Elchanan Mossel Scribes: J. Neeman, N. Truong, and S. Troxler Condorcet s theorem, the most basic jury
The n th moment M n of a RV X is defined to be E(X n ), M n = E(X n ). Andrew Dabrowski
We ll get to the in a few slides, but first some preliminaries. The tool we need to prove the is the sequence of moments of a probability distribution. We ll get to the in a few slides, but first some
GENERICITY OF INFINITE-ORDER ELEMENTS IN HYPERBOLIC GROUPS
GENERICITY OF INFINITE-ORDER ELEMENTS IN HYPERBOLIC GROUPS PALLAVI DANI 1. Introduction Let Γ be a finitely generated grou and let S be a finite set of generators for Γ. This determines a word metric on
Asymptotic Statistics-III. Changliang Zou
Asymptotic Statistics-III Changliang Zou The multivariate central limit theorem Theorem (Multivariate CLT for iid case) Let X i be iid random p-vectors with mean µ and and covariance matrix Σ. Then n (
Extension of Minimax to Infinite Matrices
Extension of Minimax to Infinite Matrices Chris Calabro June 21, 2004 Abstract Von Neumann s minimax theorem is tyically alied to a finite ayoff matrix A R m n. Here we show that (i) if m, n are both inite,
MATH 829: Introduction to Data Mining and Analysis Clustering II
his lecture is based on U. von Luxburg, A Tutorial on Spectral Clustering, Statistics and Computing, 17 (4), 2007. MATH 829: Introduction to Data Mining and Analysis Clustering II Dominique Guillot Departments
MATH 2710: NOTES FOR ANALYSIS
MATH 270: NOTES FOR ANALYSIS The main ideas we will learn from analysis center around the idea of a limit. Limits occurs in several settings. We will start with finite limits of sequences, then cover infinite
Heuristics on Tate Shafarevitch Groups of Elliptic Curves Defined over Q
Heuristics on Tate Shafarevitch Grous of Ellitic Curves Defined over Q Christohe Delaunay CONTENTS. Introduction 2. Dirichlet Series and Averages 3. Heuristics on Tate Shafarevitch Grous References In
Econometrics I. September, Part I. Department of Economics Stanford University
Econometrics I Deartment of Economics Stanfor University Setember, 2008 Part I Samling an Data Poulation an Samle. ineenent an ientical samling. (i.i..) Samling with relacement. aroximates samling without
Towards understanding the Lorenz curve using the Uniform distribution. Chris J. Stephens. Newcastle City Council, Newcastle upon Tyne, UK
Towards understanding the Lorenz curve using the Uniform distribution Chris J. Stehens Newcastle City Council, Newcastle uon Tyne, UK (For the Gini-Lorenz Conference, University of Siena, Italy, May 2005)
Homework Solution 4 for APPM4/5560 Markov Processes
Homework Solution 4 for APPM4/556 Markov Processes 9.Reflecting random walk on the line. Consider the oints,,, 4 to be marked on a straight line. Let X n be a Markov chain that moves to the right with
Piotr Blass. Jerey Lang
Ulam Quarterly Volume 2, Number 1, 1993 Piotr Blass Deartment of Mathematics Palm Beach Atlantic College West Palm Beach, FL 33402 Joseh Blass Deartment of Mathematics Bowling Green State University Bowling
General Linear Model Introduction, Classes of Linear models and Estimation
Stat 740 General Linear Model Introduction, Classes of Linear models and Estimation An aim of scientific enquiry: To describe or to discover relationshis among events (variables) in the controlled (laboratory)
Lecture 1: August 28
36-705: Intermediate Statistics Fall 2017 Lecturer: Siva Balakrishnan Lecture 1: August 28 Our broad goal for the first few lectures is to try to understand the behaviour of sums of independent random
MATH 829: Introduction to Data Mining and Analysis Principal component analysis
1/11 MATH 829: Introduction to Data Mining and Analysis Principal component analysis Dominique Guillot Departments of Mathematical Sciences University of Delaware April 4, 2016 Motivation 2/11 High-dimensional
MATH 829: Introduction to Data Mining and Analysis Graphical Models III - Gaussian Graphical Models (cont.)
1/12 MATH 829: Introduction to Data Mining and Analysis Graphical Models III - Gaussian Graphical Models (cont.) Dominique Guillot Departments of Mathematical Sciences University of Delaware May 6, 2016
IMPROVED BOUNDS IN THE SCALED ENFLO TYPE INEQUALITY FOR BANACH SPACES
IMPROVED BOUNDS IN THE SCALED ENFLO TYPE INEQUALITY FOR BANACH SPACES OHAD GILADI AND ASSAF NAOR Abstract. It is shown that if (, ) is a Banach sace with Rademacher tye 1 then for every n N there exists
4. Score normalization technical details We now discuss the technical details of the score normalization method.
SMT SCORING SYSTEM This document describes the scoring system for the Stanford Math Tournament We begin by giving an overview of the changes to scoring and a non-technical descrition of the scoring rules
Random vectors X 1 X 2. Recall that a random vector X = is made up of, say, k. X k. random variables.
Random vectors Recall that a random vector X = X X 2 is made up of, say, k random variables X k A random vector has a joint distribution, eg a density f(x), that gives probabilities P(X A) = f(x)dx Just
The Multivariate Normal Distribution. Copyright c 2012 Dan Nettleton (Iowa State University) Statistics / 36
The Multivariate Normal Distribution Copyright c 2012 Dan Nettleton (Iowa State University) Statistics 611 1 / 36 The Moment Generating Function (MGF) of a random vector X is given by M X (t) = E(e t X
MATH 829: Introduction to Data Mining and Analysis Support vector machines
1/10 MATH 829: Introduction to Data Mining and Analysis Support vector machines Dominique Guillot Departments of Mathematical Sciences University of Delaware March 11, 2016 Hyperplanes 2/10 Recall: A hyperplane
On a class of Rellich inequalities
On a class of Rellich inequalities G. Barbatis A. Tertikas Dedicated to Professor E.B. Davies on the occasion of his 60th birthday Abstract We rove Rellich and imroved Rellich inequalities that involve
On Isoperimetric Functions of Probability Measures Having Log-Concave Densities with Respect to the Standard Normal Law
On Isoerimetric Functions of Probability Measures Having Log-Concave Densities with Resect to the Standard Normal Law Sergey G. Bobkov Abstract Isoerimetric inequalities are discussed for one-dimensional
ON UNIFORM BOUNDEDNESS OF DYADIC AVERAGING OPERATORS IN SPACES OF HARDY-SOBOLEV TYPE. 1. Introduction
ON UNIFORM BOUNDEDNESS OF DYADIC AVERAGING OPERATORS IN SPACES OF HARDY-SOBOLEV TYPE GUSTAVO GARRIGÓS ANDREAS SEEGER TINO ULLRICH Abstract We give an alternative roof and a wavelet analog of recent results
Introduction to Banach Spaces
CHAPTER 8 Introduction to Banach Saces 1. Uniform and Absolute Convergence As a rearation we begin by reviewing some familiar roerties of Cauchy sequences and uniform limits in the setting of metric saces.
3. Show that if there are 23 people in a room, the probability is less than one half that no two of them share the same birthday.
N12c Natural Sciences Part IA Dr M. G. Worster Mathematics course B Examles Sheet 1 Lent erm 2005 Please communicate any errors in this sheet to Dr Worster at M.G.Worster@damt.cam.ac.uk. Note that there
Random Variables and Their Distributions
Chapter 3 Random Variables and Their Distributions A random variable (r.v.) is a function that assigns one and only one numerical value to each simple event in an experiment. We will denote r.vs by capital
arxiv:cond-mat/ v2 25 Sep 2002
Energy fluctuations at the multicritical oint in two-dimensional sin glasses arxiv:cond-mat/0207694 v2 25 Se 2002 1. Introduction Hidetoshi Nishimori, Cyril Falvo and Yukiyasu Ozeki Deartment of Physics,
Sharp gradient estimate and spectral rigidity for p-laplacian
Shar gradient estimate and sectral rigidity for -Lalacian Chiung-Jue Anna Sung and Jiaing Wang To aear in ath. Research Letters. Abstract We derive a shar gradient estimate for ositive eigenfunctions of
Notation We will be studying several counting functions related to integer artitions. For clarity, we begin with the denitions and notation. If n is a
On the Multilicity of Parts in a Random Partition Sylvie Corteel Laboratoire de Recherche en Informatique B^at 490, Universite Paris-Sud 9405 Orsay, FRANCE Sylvie.Corteel@lri.fr Carla D. Savage z Deartment
CERIAS Tech Report The period of the Bell numbers modulo a prime by Peter Montgomery, Sangil Nahm, Samuel Wagstaff Jr Center for Education
CERIAS Tech Reort 2010-01 The eriod of the Bell numbers modulo a rime by Peter Montgomery, Sangil Nahm, Samuel Wagstaff Jr Center for Education and Research Information Assurance and Security Purdue University,
Robustness of classifiers to uniform l p and Gaussian noise Supplementary material
Robustness of classifiers to uniform l and Gaussian noise Sulementary material Jean-Yves Franceschi Ecole Normale Suérieure de Lyon LIP UMR 5668 Omar Fawzi Ecole Normale Suérieure de Lyon LIP UMR 5668
Solution: (Course X071570: Stochastic Processes)
Solution I (Course X071570: Stochastic Processes) October 24, 2013 Exercise 1.1: Find all functions f from the integers to the real numbers satisfying f(n) = 1 2 f(n + 1) + 1 f(n 1) 1. 2 A secial solution
STA 2201/442 Assignment 2
STA 2201/442 Assignment 2 1. This is about how to simulate from a continuous univariate distribution. Let the random variable X have a continuous distribution with density f X (x) and cumulative distribution
738 SCIENCE IN CHINA (Series A) Vol. 46 Let y = (x 1 x ) and the random variable ß m be the number of sibs' alleles shared identity by descent (IBD) a
Vol. 46 No. 6 SCIENCE IN CHINA (Series A) November 003 The otimal design for hyothesis test and its alication in genetic linkage analysis IE Minyu (Λ Ξ) 1; & LI Zhaohai ( Π ) 1. Deartment of Statistics,
A MONOTONICITY RESULT FOR A G/GI/c QUEUE WITH BALKING OR RENEGING
J. Al. Prob. 43, 1201 1205 (2006) Printed in Israel Alied Probability Trust 2006 A MONOTONICITY RESULT FOR A G/GI/c QUEUE WITH BALKING OR RENEGING SERHAN ZIYA, University of North Carolina HAYRIYE AYHAN
= p(t)(1 λδt + o(δt)) (from axioms) Now p(0) = 1, so c = 0 giving p(t) = e λt as required. 5 For a non-homogeneous process we have
. (a (i I: P(exactly event occurs in [t, t + δt = λδt + o(δt, [o(δt/δt 0 as δt 0]. II: P( or more events occur in [t, t + δt = o(δt. III: Occurrence of events after time t is indeendent of occurrence of
1 Random Variables and Probability Distributions
1 Random Variables and Probability Distributions 1.1 Random Variables 1.1.1 Discrete random variables A random variable X is called discrete if the number of values that X takes is finite or countably
Dr. Shalabh Department of Mathematics and Statistics Indian Institute of Technology Kanpur
Analysis of Variance and Design of Exeriment-I MODULE II LECTURE -4 GENERAL LINEAR HPOTHESIS AND ANALSIS OF VARIANCE Dr. Shalabh Deartment of Mathematics and Statistics Indian Institute of Technology Kanur
Chater Matrix Norms and Singular Value Decomosition Introduction In this lecture, we introduce the notion of a norm for matrices The singular value de
Lectures on Dynamic Systems and Control Mohammed Dahleh Munther A Dahleh George Verghese Deartment of Electrical Engineering and Comuter Science Massachuasetts Institute of Technology c Chater Matrix Norms
Lecture 11. Multivariate Normal theory
10. Lecture 11. Multivariate Normal theory Lecture 11. Multivariate Normal theory 1 (1 1) 11. Multivariate Normal theory 11.1. Properties of means and covariances of vectors Properties of means and covariances
Estimation of the large covariance matrix with two-step monotone missing data
Estimation of the large covariance matrix with two-ste monotone missing data Masashi Hyodo, Nobumichi Shutoh 2, Takashi Seo, and Tatjana Pavlenko 3 Deartment of Mathematical Information Science, Tokyo
CHAPTER 2: SMOOTH MAPS. 1. Introduction In this chapter we introduce smooth maps between manifolds, and some important
CHAPTER 2: SMOOTH MAPS DAVID GLICKENSTEIN 1. Introduction In this chater we introduce smooth mas between manifolds, and some imortant concets. De nition 1. A function f : M! R k is a smooth function if
On a Markov Game with Incomplete Information
On a Markov Game with Incomlete Information Johannes Hörner, Dinah Rosenberg y, Eilon Solan z and Nicolas Vieille x{ January 24, 26 Abstract We consider an examle of a Markov game with lack of information
CSC165H, Mathematical expression and reasoning for computer science week 12
CSC165H, Mathematical exression and reasoning for comuter science week 1 nd December 005 Gary Baumgartner and Danny Hea hea@cs.toronto.edu SF4306A 416-978-5899 htt//www.cs.toronto.edu/~hea/165/s005/index.shtml