Fundamentals of Probability Theory and Mathematical Statistics Gerry Del Fiacco Math Center Metropolitan State University St. Paul, Minnesota June 6, 2016 1
Preface This collection of material was researched, assembled, written and edited during the years from 2005 to 2016. During this period of time I was employed at Metropolitan State University in St. Paul, Minnesota as an instructor and tutor of mathematics and statistics students. I previously worked in industry for decades as a computer software engineer after formal education at the University of Minnesota where I received B.A. and M.A. degrees in mathematics. An amazing diversity of life has evolved from a seeming randomness in the universe into a yearning for a cohesive existence. For this reason, an understanding of probability and statistics provides a useful perspective for contending with this dichotomy, not only in terms of endeavors in natural science and engineering but in the inevitable struggles with personal, social, economic and political matters. Gerry Del Fiacco West St. Paul, Minnesota June 6, 2016 2
Table of Contents 1 - History of Probability Theory 2 - Basic Theory of Probability 2-1 Introduction to Set Theory 2-2 Countability of Finite Sets 2-3 Cardinality of Infinite Sets 2-4 Probability Spaces 2-5 Games of Chance 2-6 Examples of Probability Calculations 2-7 Bayes Theorem 2-8 Inclusion-Exclusion Principle of Set Theory 3
3 - Advanced Topics in Probability Theory 3-1 Properties of a Random Variable 3-2 Joint Probability Distributions 3-3 Sum of Random Variables 3-4 Covariance and Correlation 3-5 Bernoulli Trials 3-6 Chebyshev's Inequality 3-7 Law of Large Numbers 3-8 Central Limit Theorem 3-9 Moment-Generating Functions 3-10 Measure Theory 3-11 Computer Simulation and Queueing Theory 3-12 Maximum or Minimum of Random Variables 4
4 - Discrete Probability Distributions 4-1 Discrete Probability Distributions 4-2 Binomial Distribution 4-3 Geometric Distribution 4-4 Hypergeometric Distribution 4-5 Poisson Distribution 5 - Continuous Probability Distributions 5-1 Continuous Probability Distributions 5-2 Normal Distribution 5-3 Chi-Square Distribution 5-4 Student T-Distribution 5-5 Fisher-Snedecor F-Distribution 6 - Simple Regression Theory 6-1 Linear Regression and Correlation 6-2 Exponential Regression 6-3 Logistic Regression 5
Appendix A - Exercises A-1 Exercises for Section 1 A-2 Exercises for Section 2 A-3 Exercises for Section 3 A-4 Exercises for Section 4 A-5 Exercises for Section 5 A-6 Exercises for Section 6 Appendix B - Tables B-1 Table of Values for Standard Normal Distribution B-2 Table of Values for Chi-Square Distribution B-3 Table of Values for Student T-Distribution B-4 Table of Values for Fisher-Snedecor F-Distribution B-5 Table of Q-Values for Tukey HSD Test Appendix C - Index of Documents Appendix D - List of References Appendix E - Elementary Statistics Calculations 6
Purpose This is a collection of papers on the fundamentals of probability theory and mathematical statistics for a college level student of mathematics. These papers could be used either for self-study or for an organized course of instruction. This material is a compendium of what a college level student with a mathematics major should learn about probability theory and mathematical statistics before entering the work industry or moving on to graduate school for further education. Probability theory provides a practical underpinning for applied mathematics in such diverse areas as medical research, public health, actuarial science, astrophysics, quantum mechanics, economics studies, climate predictions, etc. Moreover, probability theory affords a window of insight into the manner in which matter, beings and events unfold, evolve and behave in the universe about us. The orientation of these papers is to teach probability theory in a mathematically rigorous manner, supplemented by examples of the application of probability theory to inferential statistics. These papers presume that the mathematics student has completed the study of college algebra, pre-calculus, differential calculus and integral calculus. Knowledge of elementary statistics terminology and concepts also is required. 7
Examples of Statistical Inference This collection of papers includes several examples of statistical inference that illustrate the applicability of the theoretical material. These examples of statistical inference include tests of statistical significance, that is, hypothesis tests, supplemented in some cases by calculation of confidence intervals. The primary tests of statistical significance that are included in this collection of papers are listed below: Normal Distribution Chi-Square Distribution z-test of Sample Mean z-test of Sample Proportion Chi-Square Test for Goodness-of-Fit Chi-Square Test for Homogeneity Chi-Square Test for Independence Chi-Square Test for Variance z-test and Chi-Square Test of Two Proportions (Includes Fisher s Exact Test) Student T- Distribution t-test of Sample Mean t-test of Paired Samples t-test of Independent Samples Fisher-Snedecor F-Distribution One-Way ANOVA Between Subjects One-Way ANOVA With Repeated Measures Two-Way ANOVA Between Subjects F-Test for Equality of Two Variances Comparison of t-test and F-Test Linear Regression and Correlation t-test of Correlation Coefficient Analysis of Covariance (ANCOVA) 8
Format of Papers These papers exist as electronic files. They were composed using Microsoft Word 2007 (or later level). For accessibility, the Microsoft Word documents were converted to Adobe PDF format. The entire collection of files is organized into a hierarchical set of file folders that corresponds to the outline of material in the table of contents above. Appendix C consists of an index that identifies the file folder hierarchy and the files in each folder. Use of Microsoft Excel Several of these papers are accompanied by Microsoft Excel 2010 (or later level) files which are comprised of worksheets that contain the associated data, calculations and graphs that are embedded in the text of the papers. Although the papers are self-contained and can be read without accessing the Microsoft Excel worksheets, applied mathematics students should be encouraged to become adept at the use of Microsoft Excel. Microsoft Excel is used extensively in the work industry as a desktop tool and as a medium of exchange between technical and non-technical personnel. The analysis of probability distributions and the calculations that are required for the exercises in Appendix A can be performed using Microsoft Excel. Thus, the purchase and use of a programmable calculator is not required. Also, the tables of probability distributions in Appendix B are embodied in the form of Microsoft Excel spreadsheets. And, Appendix E consists of a collection of Microsoft Excel worksheets and functions that allow numerous elementary statistics calculations to be performed. Solutions to Exercises The solutions to all of the exercises in Appendix A are available in electronic form but are not physically packaged with the papers since dissemination of the solutions would diminish the value of the exercises to students. 9