PRINCIPLES OF STATISTICAL INFERENCE

Transcription

1 Advanced Series on Statistical Science & Applied Probability PRINCIPLES OF STATISTICAL INFERENCE from a Neo-Fisherian Perspective Luigi Pace Department of Statistics University ofudine, Italy Alessandra Salvan Department of Statistics University of Padua, Italy World Scientific Singapore'New Jersey London'Hong Kong

2 CONTENTS PREFACE LIST OF SYMBOLS xv xvii 1 STATISTICAL MODELS The Theory of Statistical Inference Four Paradigms of Inference Model Specification Levels of Specification Notes on the Specification of a Parametric Model Parametric Statistical Models and Likelihood General Formulation of a Statistical Model Likelihood and Related Quantities Reparameterizations Examples of Likelihood Functions Bibliographic Note Problems 29 2 DATA AND MODEL REDUCTION Introduction Statistics Distribution Constant Statistics Sufficient Statistics > Completeness Conditioning on Distribution Constant Statistics Discussion and Examples Ancillary Statistics Relevant Subsets Combinants and Pivotal Quantities 58 vii

3 viii CONTENTS 2.11 The Principle of Parameterization In variance Consequences for Parameter Estimation Consequences for Hypothesis Testing Uninformative Prior Distributions and Invariance Bibliographic Note Problems 67 3 SURVEY OF SOME BASIC CONCEPTS AND TECHNIQUES Introduction Moments, Cumulants and their Generating Functions The Moment Generating Function The Characteristic Function Convergence Results Generating Functions for Sums The Cumulant Generating Function Infinitely Divisible, Stable, Selfdecomposable Laws Multivariate Extensions Basic Notions of Asymptotic Methods Orders of Magnitude of Sequences Convergence of Sums and Extremes Orders in Probability: Examples Likelihood and First-Order Asymptotic Theory Null Asymptotic Distributions Non-null Asymptotic Distributions Robustness of Likelihood Methods Inference in the Frequency-Decision Paradigm General Framework Point Estimation Testing Hypotheses Confidence Regions": Comments on the Relations with Likelihood Inference The Empirical Distribution Function Basic Properties Nonparametric Maximum Likelihood Estimate Ill Statistical Functional Bibliographic Note Problems 118

4 CONTENTS ix 4 NUISANCE PARAMETERS AND PSEUDO-LIKELIHOODS Nuisance Parameters Data and Model Reduction with Nuisance Parameters Lack of Information: No Nuisance Parameters Lack of Information in the Presence of Nuisance Parameters Weaker Concepts of Lack of Information with Nuisance Parameters Nuisance Parameters and Reparameterizations The Notion of a Pseudo-Likelihood Marginal Likelihood: Examples Conditional Likelihood: Examples Profile Likelihood Orthogonal Parameterization and Approximate Conditional Likelihood Partial Likelihood Quasi-Likelihood Empirical Likelihood Bibliographic Note Problems EXPONENTIAL FAMILIES Introduction Exponential Families of Order Mean Value Mapping and Variance Function Multiparameter Exponential Families Definitions and Basic Results Independence, Marginal and Conditional Distributions Sufficiency and Completeness Likelihood and Exponential Families Profile Likelihood and Mixed Parameterization Procedures with Finite-Sample Optimality Properties Testing Hypotheses: One-Parameter Case Testing Hypotheses: Multiparameter Case First-Order Asymptotic Theory Curved Exponential Families Bibliographic Note. 217

5 x CONTENTS 5.12 Problems EXPONENTIAL DISPERSION FAMILIES AND GENERALIZED LINEAR MODELS Introduction Exponential Dispersion Families Parameterization (/z, a 2 ) and Convolution Properties Generalized Linear Models Likelihood and Sufficiency Quasi-Likelihood Deviance Tests Generalized Linear Models for Binary Data Bibliographic Note Problems GROUP FAMILIES Introduction Groups of Transformations Orbits and Maximal Invariants Simple Group Families Composite Group Families Inference in Simple Group Families Data and Model Reduction Likelihood and Scale and Location Families Inference in Composite Group Families Data and Model Reduction ' Marginal Likelihood Bibliographic Note Problems ASYMPTOTIC METHODS:" INTRODUCTION AND ELEMENTARY TECHNIQUES Introduction Evaluating the Accuracy of an Approximation Berry-Esseen Inequality Other Methods for Evaluating the Accuracy of a Normal Approximation for a Fixed n Chi-Squared Approximations 318

6 CONTENTS xi 8.3 Improvements on First-Order Approximations: Historical Notes Variance Stabilizing Transformations Skewness Reducing Transformations Bibliographic Note Problems ASYMPTOTIC EXPANSIONS FOR STATISTICS Index Notation Likelihood Quantities Null Moments Asymptotic Orders Some Basic Tools The Stochastic Taylor Formula Inversion of Asymptotic Series Laplace Expansion Fundamental Asymptotic Expansions Expansion of Asymptotic Bias of Variance and Other Cumulants of Expansion of 1(9) - 1(9) Expansion of E»(W) Expansion of the Profile Score Parameterization Invariance and Asymptotic Expansions Tensors Invariance of the Expansion of 1(6) - 1(9) Tensorial Behaviour of the Expansion of the Expected Value of the Profile Score Bibliographic Note Problems ASYMPTOTIC EXPANSIONS FOR DISTRIBUTIONS Generating Functions for a Standardized Sum Hermite Polynomials Edgeworth Expansion for Density Functions Edgeworth Expansion for Distribution Functions Anomalies in Edgeworth Approximations Cornish-Fisher Expansion and Polynomial Normalizing Transformation 394

7 xii CONTENTS 10.7 Saddlepoint Expansion for Density Functions Lugannani-Rice Expansion for Distribution Functions Multivariate Edgeworth and Saddlepoint Expansions Multivariate Edgeworth Expansion Multivariate Saddlepoint Expansion Mixed Expansion Asymptotic Expansions for Conditional Distributions Three Methods Exponential Families and Approximate Conditional Inference Bibliographic Note Problems LIKELIHOOD AND HIGHER-ORDER ASYMPTOTICS Introduction Approximation for the Distribution of 9: the p* Formula Null Distribution of W Effect of the Bartlett Correction Modified Versions of Z and Z p Modified Profile Likelihood Bibliographic Note Problems 463 A LAWS OF LARGE NUMBERS AND CENTRAL LIMIT THEOREMS 469 A.I Sumsofi.i.d. Random Variables 469 A.2 Sums of Independent Random Variables 471 A.3 Smooth Functions of Converging Sequences 472 A.4 Bibliographic Note 474 B ASYMPTOTIC DISTRIBUTION OF EXTREMES 475 B.I Basic Results ". 475 B.2 Bibliographic Note 476 B.3 Problems 477 C PARAMETRIC INFERENCE: BASIC TERMINOLOGY 479 C.I Point Estimation 479 C.2 Hypothesis Testing 479 C.3 Confidence Regions 480

8 CONTENTS xiii D RELATIONS BETWEEN THE FREQUENCY-DECISION AND FISHERIAN PARADIGMS 483 REFERENCES 489 AUTHOR INDEX 521 SUBJECT INDEX 527