Foundations of Probability and Statistics

Foundations of Probability and Statistics William C. Rinaman Le Moyne College Syracuse, New York Saunders College Publishing Harcourt Brace College Publishers Fort Worth Philadelphia San Diego New York Orlando Austin San Antonio Toronto Montreal London Sydney Tokyo

Table of Contents Chapter 1 Random Experiments and Probability 1 1.1 Introduction 1 1.2 Set Theory 3 1.3 Random Experiments 12 1.4 The Axioms of Probability 17 1.5 Counting Elementary Events 27 1.6 Probabilities for Finite Sample Spaces with Equally Likely Outcomes 41 Chapter Summary 53 Review Exercises 56 Chapter 2 Conditional Probability and Independence 59 2.1 Introduction 59 2.2 Conditional Probability 61 2.3 Bayes'Rule 71 2.4 Independence 77 Chapter Summary 92 Review Exercises 92 Chapter 3 Random Variables 95 3.1 Introduction 95 vü 3.2 The Distribution Function 99

viii Table of Contents 3.3 Discrete Random Variables 108 3.4 Continuous Random Variables 114 3.5 Random Vectors 121 3.6 Independent Random Variables 139 3.7 Conditional Distributions 145 Chapter Summary 149 Review Exercises 152 Chapter 4 Expectation 155 4.1 Introduction 155 4.2 Expected Value 156 4.3 Expectation of a Function of a Random Variable 162 4.4 Moments 173 4.5 Moment Generating Functions 189 4.6 Conditional Expectation 197 Chapter Summary 203 Review Exercises 204 Chapter 5 Probability Distributions 207 5.1 Introduction 207 5.2 Hypergeometric Distribution 208 5.3 Distributions Based on Independent Bernoulli Trials 211 5.4 Multinomial Distribution 218 5.5 Poisson Distribution 223 5.6 Uniform Distribution 228 5.7 Normal Distribution 232 5.8 Gamma Distribution 239 5.9 Other Continuous Distributions 244 5.9.1 Weibull Distribution 244 5.9.2 Beta Distribution 245 5.9.3 Cauchy Distribution 246 5.9.4 Double Exponential Distribution 247 5.9.5 Logistic Distribution 248 5.10 Bivariate Normal Distribution 250

Table of Contents ix Chapter Summary 253 Review Exercises 254 Chapter 6 Distributions of Functions of Random Variables 257 6.1 Introduction 257 6.2 Transformations of Discrete Random Variables 257 6.3 The Distribution Function Method 262 6.4 The Transformation of Variables Method 265 6.5 Moment Generating Function Method 270 6.6 Sums of Independent Random Variables 274 6.7 Distribution of Functions of Several Random Variables 277 Chapter Summary 286 Review Exercises 288 Chapter 7 Limit Theorems 291 7.1 Introduction 291 7.2 Convergence in Probability 292 7.3 Central Limit Theorem 308 7.4 More on Convergence in Distribution 315 Chapter Summary 319 Review Exercises 322 Chapter 8 Statistical Models 325 8.1 Introduction 325 8.2 Populations and Random Sampling 327 8.3 Numerical Summaries 332 8.3.1 Sample Mean 333 8.3.2 Sample Median 334 8.3.3 Sample Variance 334 8.3.4 Median Absolute Deviation 335 8.3.5 Percentile-Based Measures of Dispersion 335 8.3.6 Sample Skewness 337 8.3.7 Sample Kurtosis 338 8.3.8 An Example 338

x Table of Contents 8.4 Graphical Summaries 341 8.4.1 Quantile-Quantile Plots 342 8.4.2 Histograms 345 8.4.3 Stem-and-Leaf Plots 346 Chapter Summary 349 Review Exercises 350 Chapter 9 Sampling Distributions 9.1 Introduction 351 9.2 Distributions Based on Samples from Normal Populations 9.3 Independence of X and s 2 359 9.4 Order Statistics 362 Chapter Summary 377 Review Exercises 378 Chapter 10 Point Estimation 10.1 Introduction 381 10.2 Maximum Likelihood Estimation 382 10.3 Method of Moments 390 10.4 Properties of Estimators 394 10.4.1 Unbiasedness 394 10.4.2 Efficiency 398 10.4.3 Sufflciency 408 10.4.4 Minimal Sufficient Statistics 417 10.4.5 Completeness 419 10.5 Large Sample Properties of Estimators 430 10.5.1 Consistency 430 10.5.2 Asymptotic Efficiency 432 10.6 Bayes Estimators 434 10.7 Robust Estimation 441 Chapter Summary 444 Review Exercises 449

Table of Contents xi Chapter 11 Confidence Intervals 451 11.1 Introduction 451 11.2 Pivotal Quantity Method 453 11.3 Method Based on Sampling Distributions 463 11.4 Large Sample Confidence Intervals 468 11.5 A Nonparametnc Confidence Interval 472 Chapter Summary 474 Review Exercises 476 Chapter 12 Hypothesis Testing 479 12.1 Introduction 479 12.2 Types of Errors 481 12.3 Testing Simple Hypotheses 489 12.4 Uniformly Most Powerful Tests 497 12.5 Likelihood Ratio Tests 509 12.6 Chi-Square Tests 528 12.7 Large Sample Tests 535 12.8 Sequential Probability Ratio Test 540 12.9 Nonparametric Tests 546 12.9.1 Sign Test 547 12.9.2 Wilcoxon Signed Rank Test 551 12.9.3 Wilcoxon Rank Sum Test 553 Chapter Summary 558 Review Exercises 564 Chapter 13 Regression and Correlation 569 13.1 Introduction 569 13.2 Simple Linear Regression 570 13.3 Matrix Calculus 589 13.4 Multiple Regression 595 13.5 Correlation 607 13.6 Nonparametric Methods 615 Chapter Summary 622 Review Exercises 630

- xii Table of Contents Chapter 14 Appendix A Analysis of Variance 14.1 Introduction 633 14.2 Experimental Design 634 14.3 One-Way Analysis of Variance 14.4 Two-Way Analysis of Variance 14.5 Estimation 667 14.6 Nonparametric Methods 673 Chapter Summary 681 Review Exercises 687 Statistical Tables 641 652 633 A.1 I Binomial Distribution A.l II Poisson Distribution A.7 III Normal Distribution A.l3 IV Chi-Square Distribution A.14 V Student's t Distribution A.l 5 VI F Distribution A.l6 VII Wilcoxon Signed Rank Test A.20 VIII Wilcoxon Rank Sum Test A.27 IX Spearman's Rho A.35 X Kruskal-Wallis Test A.38 XI Friedman Test A.53 Appendix B Probability Distributions Discrete Random Variables A.59 Continuous Random Variables A.61 A.59 Appendix C Bibliography A.63 Appendix D Answers to Selected Odd-Numbered Exercises A.65 Index J LI

Random Experiments and Probability Most people know intuitively what the word "probability" means. Many of us have encountered probabilities in our daily activities. For example, if we flip a coin it is our understanding that heads should result about half the time. When a local weather forecaster states that there is a 40 percent chance for rain tomorrow we have a general understanding that what is meant is that, given the current conditions, rain may occur 4 times out of 10. If we play bridge, we may draw on past experience to assess the likelihood that the cards held by our opponents have been distributed in a certain way in order to plan how we play a current hand. In fact, anyone who is consistently successful at games involving chance such as poker and backgammon is well aware of the relative frequency of the various possible outcomes in the play of the game. The situations mentioned above have some things in common. In each we face a Situation that is, at least in principle, well defined and repeatable. The possible outcomes are known, but the particular outcome that will take place this time cannot be predicted. What probability attempts to do is to determine a numerical value that teils what proportion of the time each possibility will occur. One could correctly argue that the weather example is not like the others. It does not involve chance occurrences but rather represents a less than complete understanding of the physics of the atmosphere. This is the case in a number of areas where probability may be applied. Many physical situations are so complicated that it is not feasible to develop a mathematical model that accounts for all of the variables. In such cases it is common to develop modeis that use the main variables and then lump together the unused variables in a term that represents the unpredicted part of the model called the "noise." The noise in the model is what injects randomness into predictions.