HANDBOOK OF APPLICABLE MATHEMATICS

Transcription

1 HANDBOOK OF APPLICABLE MATHEMATICS Chief Editor: Walter Ledermann Volume II: Probability Emlyn Lloyd University oflancaster A Wiley-Interscience Publication JOHN WILEY & SONS Chichester - New York - Brisbane - Toronto

2 INTRODUCTION 1. INTRODUCTION: PROBABILITY AND STATISTICS 2. BASIC CONCEPTS OF PROBABILITY Randomness Frequency tables Probability: intuitive ideas 3. THE CALCULUS OF PROBABILITY 3.1. Sample space. Events and sets 3.2. Axioms of probability for discrete sample Spaces 3.3. Randomization. Fundamental sampling theorem 3.4. Relationships between events Negation Conjunction P(AB) in terms of P(A) and P{B): Disjoint events Disjunction Note on notation Conjunctions and disjunctions involving more than two events Implication Statistical independence Some applications ofthe concept of independence Reduction in dimensionality of sample space Axiom on independence-conferring property of randomization Sampling with replacement from a finite population 3.7. Bernoulli trials: sampling with replacement from a finite population containing only two kinds of articles Bernoulli trials in other contexts including (as an approximation sampling without replacement. The Bernoulli process 3.9. Non-independent events and conditional probability Conditional probability.... v

3 Conditional probability and a factorization rule for P(A andb) Conditional probability and independence Conditional probabilities are probabilities Manipulative properties of conditional probabilities Conditional probability statements involving more than two events RAN DOM VARIABLES AND THEIR REALIZATIONS; PROB ABILITY DISTRIBUTIONS 4.1. Random variables Random variables as functions The probability distribution of integer-valued random variables Distribution specified by p.d.f Distribution specified by cd.f Multivariate random variables: independent random variables 4.5. The calculus of random variables: addition Introduction Addition of random variables Constants regarded as random variables Transformations of random variables A FIRST CATALOGUE OF DISCRETE PROBABILITY DISTRI BUTIONS {INTEGER-VALUED VARIABLES) The discrete uniform distribution Distributions arising in connection with the Bernoulli sequence Bernoulli trial variables: trials with success-parameter p Otrials') The binomial distribution with n trials and success parameter p (the binomial (n, p) distribution) The geometric distribution. 6 7 (i) first version (ii) second version of the geometric distribution The negative binomial distribution (the Pascal distribution) The hypergeometric distribution The Poisson distribution The Poisson distribution as approximation to the binomial and to the hypergeometric distribution Simulation: generating realizations of a discrete random variable 11 INDEPENDENT AND NON-INDEPENDENT RANDOM VARI ABLES AND THEIR JOINT DISTRIBUTIONS, B1VARIATE AND MULTIVARIATE; MARGINAL AND CONDITIONAL DISTRI BUTIONS Bivariate distributions

4 Bivariate random variables and their Joint p.d.f Bivariate distributions and their Joint c.d.f Bivariate frequency tables 6.3. Marginal distributions Introduction Marginal distributions in the case of independence 6.4. An important multivariate distribution: the multinomial and a special case: the trinomial The trinomial distribution Multinomial distributions and their marginals 6.5. Conditional probabilities and conditional distributions 6.6. Bivariate and multivariate probabilities expressed in terms of conditional probabilities, and the special case of independence Introduction The special case of independence. Distribution of a sample Truncated distributions The 'residual waiting time' property ofthe geometric distribution The interval between successive failures in a Bernoulli process vu THE ADDITION OF RANDOM VARIABLES: CONVOLUTION Introduction Non-negative variables Variables bounded above and below Examples of the convolution Operation The sum of independent /»-binomials is p binomial The sum of geometrically distributed variables EXPECTATION (EXPECTED VALUE) Motivation and definition: examples 8.2. Expectation ofa sum Expectation ofa constant Expectation ofa linear function of random variables Expectation ofa non-linear function ofa random variable 8.6. Expectation of a product of independent variables 8.7. Variables that do not have afinite expectation A minimizing property of E(X) Conditional expectation VARIANCE, COVARIANCE, AND HIGHER MOMENTS Variance: motivation, and definition of sample variance Standard deviation and variance ofa random variable ('population values') Definitions and examples 131

5 VUl Effect of linear transformation Variance of a sum of independent variables Variance of scaled sum of independent variables Variance of a linear function of independent variables Coefficient of Variation Random variables that do not have afinite variance 9.4. Scaling, standardization and skewness Chebychev's inequality. Markov's inequality 9.6. Covariance. Variance of a linear function of random variables Uncorrelated variables Definition of covariance Properties of the covariance Operator Uncorrelated variables Covariance of one linear function with another: matrix form Variance of a linear function The variance-covariance inequality The correlation coefficient Definition and basic properties The correlation coefficient and linear transformations The correlation coefficient as a measure of Statistical associ ation Approximations for the expectation, variance and covariance of nonlinear functions Introduction Approximations for E{h(X)} Approximations for var {h(x)} Approximations for cov {g(xx), h(x 2 )} Shape: skewness and kurtosis Geometrical motivation Coefficient of skewness of linear transformations and sums of random variables Coefficient of skewness of a sum of independent and iden tically distributed variables Kurtosis Moments Conditional variance and conditional moments. 10. CONTINUOUS DISTRIBUTIONS Introduction Basic properties of the p.d.f. and the c.d.f An intuitive interpretation of the graph of a p.d.f The mode of a continuous p.d.f Some examples of continuous random variables The continuous uniform distribution n Uniformly distributed particles The exponential distribution

6 IX The cumulative distribution function and the probability density function ofa continuous random variable The cumulative distribution function The probability density function Fractiles and percentiles Measures of location and scale Expected value and variance of a continuous random variable Other measures of location and scale Joint distributions: independence Expected value of a sunt Transformations Convolutions A CATALOGUE OF CONTINUOVS PROBABILITY DISTRI BUTIONS The continuous uniform distribution Standard and other forms Convolutions of uniformly distributed variables The (negative) exponential distribution Standard and other forms Convolutions and special properties 77 J. The gamma family Standard and scaled forms of the p.d.f. of the gamma distri bution Relation between gamma and exponential distributions convolutions. Erlangian distributions Occurrence and applications The Normal (Gaussian) distribution Standard form Occurrence and applications: Introduction to the central limit theorem The Normal family The value of a certain definite integral Convolutions of Normal distribution Numerical values of Normal probabilities. Use of tables The Normal distribution as an approximation to the binomial Normal probability graph paper ('arithmetic probability paper') The Normal distribution and errors of measurement The Normal distribution and errors of computation The Chi-squared distribution The log-normal distribution The Beta distribution Standard form

7 X Linearly transformed Beta distribution Relation of Beta distribution with other distributions The Cauchy distribution The Pareto distribution The Weibull distribution The Logistic distribution The Powered Gamma distribution (Kritskii-Menkel) The Pearson System Simulation: generating realizations of continuous random variables GENERATING FUNCTIONS Introduction The probability generating function (p.g. f.) Definition of the p.g.f The p.g.f. as expectation Convolutions Tail probabilities Expectation obtained from the p.g.f Variance obtained from the p.g.f Extended definition of the p.g.f: variables taking positive and negative values. The p.g.f. of R T Probability generating function of a discrete random variable whose values are not necessarily integers The p.g.f. of a multivariate distribution The moment generating function (m.g.f.) Definition Major properties Bivariate and multivariate m.g.f.'s The Laplace Transform (L.T) The characteristic function Introduction and definition Linear transformations Principal properties Bivariate and multivariate c.f.'s The second characteristic function {cumulant generating functiori) Definition Another version of the c.g.f Relation between cumulants and moments Effect of shift of origin Cumulants of a sum of random variables The factorial moment generating function (f.m.g. f.) MULTIVARIATE CONTINUOUS DISTRIBUTIONS Joint, marginal and conditional distributions; independence Introduction: Joint distributions

8 XI Marginal distributions The moments of a multivariate distribution Conditional distributions Regression Independence: the distribution ofa sample Multivariate moment generating functions Multivariate transformations Introduction Invertible transformations Linear Operations on jointly distributed random variables Vector variables, expectation vector, dispersion matrix Linear transforms More on variances and covariances (i) The cross-covariance matrix (ii) Dispersion matrix of a sum The bivariate Normal distribution The multivariate Normal (MVN) distribution The MVN density function The MVN distribution and independence Moment generating function of the MVN distribution The marginal distribution of the MVN (i) Normal marginals do not imply a Multivariate Normal distribution (ii) Marginal distribution of the MVN derived in terms of the moment generating function The conditional distributions of the MVN Summary of properties of the bivariate and trivariate Normal distribution (i) Bivariate Normal 281 (ii) Trivariate Normal (iii) Partial Correlation coefficient The Joint distribution of a set of linear functions of MVN variables 284 Independence and Orthogonal transformations in the MVN The distribution of a certain quadratic form An example ofa non-normal multivariate distribution. The Dirichlet distribution: a multivariate beta Linear regression ('multiple regression') and the multiple correlation coefficient MISCELLANEOUSTOPICS ON DISTRIBUTIONS Distributions that are part continuous, part discrete Mixtur es of distributions Compound distributions: the sum ofa random number of random variables Distribution of S N 292

9 xii Mean and variance of S N Use of the Probability Generating Function for Compound distributions Moment generating function of S N 15. ORDER STATISTICS Introduction The distribution of individual order statistics when the underlying distribution is continuous (i) Largest value (ii) Smallest value... (iii) A general value. (iv) Order statistics of the exponential and the logistic distributions The Order statistics of discrete variables The Joint distribution of several order statistics in samples from a continuous distribution Median, ränge and extreme values for continuous distributions '" Extreme-value" distributions MISCELLANEOUS TOPICS INVOLVING CONDITIONAL PROBABILITY The conditioned variable X \ y Theorem of Total Probability The conditional expectation theorem Bayes' Theorem Introduction: Bayes' Theorem with binomial sampling Bayes' Theorem with general sampling from a population selected from a finite or countable set Bayes' Theorem for a continuous parameter LAWS OFLARGE NUMBERS. STABLE DISTRIBUTIONS The {weak) law of large numbers Strong law of large numbers Central Limit Theorem Stable distributions STOCHASTIC PROCESSES: INTRODUCTION AND EX- AMPLES Introduction: The State space and parameter space Univariate random variables Multivariate random variables Discrete-state or continuous-state stochastic processes with discrete parameter Continuous-parameter stochastic processes Other classifications of stochastic processes Dimensionality

10 xiii Sequence of sums of random variables The branching process General description Mean size of the nth generation Probabilities of ultimate extinction and of indefinite survival Population size after a large number of generations Effect of starting from more than a Single initial particle Applications to survival of family names The Gambiers Ruin Introduction The probability of ruin Expected duration of play Distribution of duration of play The Random Walk Introduction: the State space and the parameter space Simple random walk. First passages. Absorbing barriers (i) Unrestricted simple random walk. (ii) First passages..... (iii) First passages by the method of unit descents (iv) An improper random variable (v) First passage time by the method of images: (geometric version) (vi) The method of images (analytic version). (vii) First returns in an unrestricted random walk (viii) Time to absorption, by the method of images (ix) First passages when there are two absorbing barriers More general one-dimensional random walks. Wald's identity Diffusion 19. PROCESSES IN DISCRETE TIME: MARKOV CHAINS Generalities on stochastic processes General dependence, independence and Markovian dependence Discrete-state Markov Chains The s-step transition probability The distribution of X t The equilibrium distribution, the stationary distribution and ergodic chains Classification of states and chains Communicating states Irreducible chains Periodic and aperiodic chains Irreducible chains and equilibrium distributions Further properties of Markov Chains. Closed sets and ab sorbing states: transient states The correlation function of a Markov Chain Markov Chains of the second (or higher) order

11 XIV Continuous-state Markov Chains CONTINUOUS-T1ME PROCESSES The Poisson process General definition: the one-dimensional process Waiting times from a fixed origin Interval between consecutive events Superposition of independent Poisson processes Clustering Non-homogeneous Poisson processes The homogeneous Poisson process in two or more dimensions Nearest neighbours Spatio-temporal Poisson fields Projections of multi-dimensional Poisson processes Birth processes The two-state process The Birth-and-Death process General description Statistical equilibrium Special case: 'Linear growth' The simple queue Markov Chains with continuous Urne parameters RENEWALS Integer-valued random variables. Introduction, Definition, Examples Fundamental Theorems on recurrence-time distributions Number of occurrences The Markov Chain as a renewal process Modified renewal processes Renewal Theory in continuous time COMPLEMENTS ON STOCHASTIC PROCESSES. ORDER PROCESSES Definitions The covariance function Definitionss Properties of the auto-covariance function Spectral density Introduction to stochastic analysis Convergence Continuity Differentiability.... SECOND BIBLIOGRAPHY INDEX