Exercise Manual in Probability Theory

Transcription

1 Exercise Manual in Probability Theory

2 Mathematics and Its Applications (East European Series) Managing Editor: M. l-iazewinkel Centre for Mathematics and Computer Science, Amsterdam, The Netherlands Editorial Board A. BIAl.YNICKI-BIRULA, Institute of Mathematics, Warsaw University, Poland H. KURKE, Humboldt University, Berlin, a.d.r. J. KURZWEIL, Mathematics Institute, Academy of Sciences, Prague, Czechoslovakia L. LEINDLER, Bolyai Institute; Szeged, Hungary L. LOV Asz, Bolyai Institute, Szeged, Hungary D. S. MITRINOVIC, University of Belgrade, Yugoslavia S. ROLEWICZ, Polish Academy of Sciences, Warsaw, Poland BL. H. SENDOV, Bulgarian Academy of Sciences, Sofia, Bulgaria I. T. TODOROV, Bulgarian Academy of Sciences, Sofia, Bulgaria H. TRIEBEL, University of lena, a.d.r.

3 J. Stoyanov, 1. Mirazchiiski, Z. Ignatov, and M. Tanushev Institute of Mathematics, Sofia, Bulgaria Exercise Manual in Probability Theory Edited by K. Kocherlakota Kluwer Academic Publishers Dordrecht / Boston / London

4 Library of Congress Cataloging in Publication Data Exercise manual in probability theory. (Mathematics and its applications. East European series) Bibliography: p. Includes index. 1. Probabi1ities--Prob1ems, exercises, etc. I. StoLanov, Iordan. II. Kocher1akota, K. III. Series. QA E ' ISBN-13: DOl: / e-isbn-13: Published by Kluwer Academic Publishers, P.O. Box 17,3300 AA Dordrecht, The Netherlands. Kluwer Academic Publishers incorporates the publishing programmes of D. Reidel, Martinus Nijhoff, Dr W. Junk and MTP Press. Sold and distributed in the U.S.A. and Canada by Kluwer Academic Publishers, 101 Philip Drive, Norwell, MA 02061, U.S.A. In all other countries, sold and distributed by Kluwer Academic Publishers Group, P.O. Box 322, 3300 AH Dordrecht, The Netherlands. This English edition is a revised and updated version of 'Rukovodstvo za uprazhnenia po teoria na veroyatnostite' (1985) Translated from the Bulgarian by L. Boneva, O. Enchev. V. Kaishev, and L. Markov All Rights Reserved 1989 J. Stoyanov, I. Mirazchiiski, Z. Ignatov, and M. Tanushev c/o JUSAUTOR, Sofia, Bulgaria English edition 1989 by Kluwer Academic Publishers Original 1985 by Nauka i Izkustvo, Sofia, Bulgaria Softcover reprint of the hardcover 1st edition 1989

5 Table of Contents Series Editor's Preface Preface Basic Notations and Abbreviations CONTENTS CHAPTER 1. ELEMENTARY PROBABILITIES 1. Combinatorics 1 2. Events and Relations among Them 9 3. Classical Definition of Probability Conditional Probability. Independence of Events Probability of a Sum of Events. Formula for Total Probability. Bayes' Formula Urn Models. Polya Urn Model Geometric Probabilty Bernoulli Trials. Binomoal and Multinomial Distributions Discrete Random Variables and Their Characteristics Normal and Poisson Approximations for the Binomial Distribution 65 CHAPTER 2. PROBABILITY SPACES AND RANDOM VARIABLES General Definition of Probability and a-algebra of Events Random Variables and Integration Conditional Probability, Independence and Martingales Product of Measurable Spaces and Probabilities on Them 90 CHAPTER 3. CHARACTERISTICS OF RANDOM VARIABLES Distribution Function Multivarite Distributions and Functions of Random Variables Expectation, Variance and Moments of Higher Order Generating Functions and Characteristic Functions Infinitely Divisible and Stable Distributions Conditional Distributions and Conditional Expectation Inequalities for Random Variables 147 CHAPTER 4. LIMIT THEOREMS Types of Convergence for Sequences of Random Variables Laws of Large Numbers Central Limit Theorem and Related Topics 168 SOLUTIONS, HINTS, AND ANSWERS 177 Table 1 (Normal distribution) 346 Table 2 (poisson distribution) 347 References 348 Subject Index 350 vii ix xi

6 SERIES EDITOR'S PREFACE Approach your problems from the right end and begin with the answers. Then one day, perhaps you will find the final question. 'The Hermit Clad in Crane Feathers' in R. van Gulik's The Chinese Maze Murders. It isn't that they can't see the solution. It is that they can't see the problem. G.K. Chesterton. The Scandal of Father Brown 'The point of a Pin'. Growing specialization and diversification have brought a host of monographs and textbooks on increasingly specialized topics. However, the "tree" of knowledge of mathematics and related fields does not grow only by putting forth new branches. It also happens, quite often in fact, that branches which were thought to be completely disparate are suddenly seen to be related. Further, the kind and level of sophistication of mathematics applied in various sciences has changed drastically in recent years: measure theory is used (non-trivially) in regional and theoretical economics; algebraic geometry interacts with physics; the Minkowsky lemma, coding theory and the structure of water meet one another in packing and covering theory; quantum fields, crystal defects and mathematical programming profit from homotopy theory; Lie algebras are relevant to filtering; and prediction and electrical engineering can use Stein spaces. And in addition to this there are such new emerging subdisciplines as "experimental mathematics", "CFD", "completely integrable systems", "chaos, synergetics and large-scale order", which are almost impossible to fit into the existing classification schemes. They draw upon widely different sections of mathematics. This programme, Mathematics and Its Applications, is devoted to new emerging (sub)disciplines and to such (new) interrelations as exempla gratia: - a central concept which plays an important role in several different mathematical and/or scientific specialized areas; - new applications of the results and ideas from one area of scientific endeavour into another; - influences which the results, problems and concepts of one field of enquiry have and have had on the development of another. The Mathematics and Its Applications programme tries to make available a careful selection of books which fit the philosophy outlined above. With such books, which are stimulating rather than definitive, intriguing rather than encyclopaedic, we hope to contribute something towards better communication among the practitioners in diversified fields. Because of the wealth of scholarly research being undertaken in the Soviet Union, Eastern Europe, and Japan, it was decided to devote special attention to the work emanating from these particular regions. Thus it was decided to start three regional series under the umbrella of the main MIA programme. Possibly the best way to learn any topic at all is to work in it, to do it. According to some this is the only way. With that I do not agree, especially in the earlier stages, the first years of study; in that period the method is to slow and laborious. However for those who have had basic mathematical training and especially for those who are already professionals in another branch of mathematics there are few better tools to acquire skills and knowledge than a book full of problems which also contains just enough basic definitions and theory to make it usuable as a standard-alone. VII

7 viii SERIES EDITOR'S PREFACE This is precisely such a book for probability and it will surely help many specialists from other fields to acquire active knowledge of this large and immensely useful subject. The unreasonable effectiveness of mathematics in science... Eugene Wigner Well, if you know of a better 'ole, go to it. Bruce Bairnsfather What is now proved was once only imagined. As long as algebra and geometry proceeded along separate paths, their advance was slow and their applications limited. But when these sciences joined company they drew from each other fresh vitality and thenceforward marched on at a rapid pace towards perfection. Joseph Louis Lagrange. William Blake Antibes, June 1988 Michiel Hazewinkel

8 PREFACE This book is a collection of 777 exercises covering basic topics of Probability theory. The book is intended mainly to those studying and teaching mathematics at university level, but can be used also by every one who teaches or studies elements of Probability theory. The material is organized and presented in a way which makes the book suitable for self-education as well. Ilany of the widely known examples and exercises which form the so-called 'probability classics' are included, as well as several new and original ones. Some of the exercises are elementary but other ones require serious probabilistic reasoning. The exercises of higher degree of difficulty are asterisked. The material is distributed into 24 sections which are grouped in the following four chapters: 1. Elementary probabilities. 2. Probability spaces. Random variables. 3. Characteristics of random variables. 4. Limit theorems. Each section comprises introductory notes (basic definitions and statements), illustrative examples and formulation of the exercises followed by complete solutions, hints or answers. In order to facilitate the solution of some of the problems we have included two tables (Normal and Poisson distributions). The list of references consists of titles which have been used and could be recommended for further work. The present book is a result of our nearly twenty years lona experience in teaching Probability theory at the University of Sofia. The first Bulgarian edition appeared in 1976 and in 1982 it was translated into Polish. The present English translation is based on the second improved Bulgarian edition (1985), but we have enlarged the introductory notes to the sections and added new illustrative examples. The book as a whole is a result of the joint work of the authors; however, each of them was responsible for the following sections: I. Mirazchiiski: 1, 2, 3, 4, 5, 8; M. Tanushev: 6, 7, 11, 12, 13, 14; Z. I gna tov : 15, 16, 17, 18, 20; J. S toyanov : 9, 10, 1 '), 21, 22, 23, 24. The coordination when preparing both Bulgarian editions as well as the English translation of the book was carried out by J. Stoyanov. We are very pleased that the present English edition is planned to appear in 1988 when the University of Sofia is going to celebrate its 100 anniversary. Our special thanks are addressed to Prof. K. Kocherlakota, the editor of the translation. His very useful comments and suggestions helped us to avoid several omissions of different kind thus improving both the content and the style of the presentation. Finally, it is our pleasant duty to thank Kluwer Academic Publishers for their collaboration. Sofia, Harch 1987 The authors ix

9 BASIC NOTATIONS AND ABBREVIATIONS (n, 1\, P) ]2, B A W rl A, ;: r 0(r) P (A) p(aib) IR n 13 n I;, 1l, 1; EE; VE; E{E;IF} F f probability space random events (events) event complementary to A simple event (elementary outcome) set of elementary outcomes (sample space) algebras or 0-algebras of events class of events a-algebra generated by r probability of the event A conditional probability of A given B n-dimensional Euclidean space Borel a-algebra in IR n random variables expectation (mean value) of E; variance of E; conditional expectation of E; given F distribution function: F(x) = p{w : E;(w) < x}, x 1R1 density: F(x) = J:oo f(u) du, x 1R1 convolution of Fl and F 2 : (F 1 *F 2 ) (x) convolution of f1 cpo )In P (k) n B(n, p) PtA) N(a, 0 2 ) (f *f ) (x) = Jr'" 1 2 _00 characteristic function: (t) = E{exp(itE;)}, t standard normal distribution function: 1 JX _u 2 /2 cp(x) =72TI e' du _00 1 _x2 /2 standard normal density: cpo (x) = I2TI e. 1R1 number of successes in Bernoulli scheme (n, p) n k n-k binomial probability: Pn(k) = p{)jn k} = (k)p (1 p) set of binomially distributed random variables with parameters nand p set of Poisson distributed random variables with parameter A set of normally distributed random variables with parameters a and 0 2 xi

10 xii E(A) L r r.v. d.f. ch.f. g.f. a.s. (P-a.s. ) d -> p -> a.s. ~ L r ---> F(X O + 0) F(X O - 0) rea) B(a, S) la' 1A [xl Re z Res g(zo) 0 BASIC NOTATIONS AND ABBREVIATIONS set of exponentially distributed random variables with parameter A space of random variables with integrable r-th power random variable distribution function characteristic function generating function almost sure almost sure with respect to the probability P convergence in distribution convergence in probability convergence almost surely convergence in L -sense r limit of F(x) as x ~ limit of F(x) as x Xo Xo Euler's gamma function: rea) Euler's beta function: B(a, S) indicator of the set A: 1A ((;J) w E A ( a-1 x J O the integer part of the real number x the real part of the complex number z residue of the function g(z) at z = Zo end -x e dx r1 a-1 S-l J x (1 - x) dx o 1 if w E A and 1 A (W) o if