ON THE STRONG LAW OF LARGE NUMBERS FOR WEIGHTED SUMS OF RANDOM ELEMENTS IN BANACH SPACES

Transcription

1 ON THE STRONG LAW OF LARGE NUMBERS FOR WEIGHTED SUMS OF RANDOM ELEMENTS IN BANACH SPACES By YUAN LIAO A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA 2012

2 c 2012 YUAN LIAO 2

3 To my teachers with gratitude To my parents with love 3

4 ACKNOWLEDGMENTS First and most importantly, I must convey my sincerest gratitude to my Ph.D advisor, Professor Andrew Rosalsky, for his invaluable guidance and constant support throughout my graduate studies. This dissertation would not have been possible without his step-by-step guidance. He always generously shares his ideas and makes great effort to explain them in the clearest way possible. I feel very fortunate to get to know him. He is an amiable mentor full of enthusiasm for probability theory. He is always patient to correct the faults in my work and provide the most informative and inspiring feedback. I must admit that I have learned a lot from his meticulous academic attitude. Next, I would like to thank everyone else on my supervisory committee: Distinguished Professor Malay Ghosh, Dr. Kshitij Khare, and Dr. Amy Cantrell. I am grateful for all of their support and help. Finally, I would extend my earnest thanks to my parents, who have always been confident and pride of me and encouraging me to chase my dreams. They have always and forever been my inspiration! 4

5 TABLE OF CONTENTS page ACKNOWLEDGMENTS LIST OF FIGURES ABSTRACT CHAPTER 1 INTRODUCTION Strong Law of Large Numbers for Random Variables Strong Law of Large Numbers for Banach Space Valued Random Elements Motivation and Organization of Dissertation PRELIMINARIES: DEFINITIONS, LEMMAS, AND NOTATION Basic Concepts of Banach Spaces Probability in Banach Spaces Useful Lemmas STRONG LAWS OF LARGE NUMBERS IN RADEMACHER TYPE p (1 p 2) BANACH SPACES FOR INDEPENDENT SUMMANDS Objective Main Results STRONG LAWS OF LARGE NUMBERS FOR RANDOM ELEMENTS IN GENERAL BANACH SPACES IRRESPECTIVE OF THEIR JOINT DISTRIBUTIONS Objective Main Results FUTURE RESEARCH AND CONCLUSIONS Future Research Conclusions REFERENCES BIOGRAPHICAL SKETCH

6 Figure LIST OF FIGURES page 2-1 Expected Value of a Random Element in L p (R), 1 p <

7 Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy ON THE STRONG LAW OF LARGE NUMBERS FOR WEIGHTED SUMS OF RANDOM ELEMENTS IN BANACH SPACES Chair: Andrew Rosalsky Major: Statistics By YUAN LIAO May 2012 Let {V n, n 1} be a sequence of random elements in a real separable Banach space and suppose that {V n, n 1} is stochastically dominated by a random element V. Let {a n, n 1} and {, n 1} be real sequences with 0 <. The main results are strong laws of large numbers (SLLNs) obtained for the following two broad cases; the results are new even when the underlying Banach space is the real line. (i) Conditions are provided under which {a n (V n EV n ), n 1} obeys a general SLLN of the form n i=1 a i(v i EV i )/ 0 almost certainly where the {V n, n 1} are independent. The underlying Banach space is assumed to satisfy the geometric condition that it is of Rademacher type p (1 p 2). Special cases include results of Woyczyński (1980), Teicher (1985), Adler, Rosalsky, and Taylor (1989), and Sung (1997). (ii) Conditions are provided under which {a n V n, n 1} obeys a general SLLN of the form n i=1 a iv i / 0 almost certainly irrespective of the joint distributions of the {V n, n 1}. No geometric conditions are imposed on the underlying Banach space. The results are general enough to include as special cases results of Petrov (1973), Teicher (1985), Sung (1997), and Rosalsky and Stoica (2010). Numerous examples are provided which illustrate, compare, or demonstrate the sharpness of the results. 7

8 CHAPTER 1 INTRODUCTION 1.1 Strong Law of Large Numbers for Random Variables Probability theory, as a mathematical discipline concerned with the analysis of random or chance phenomena, has developed not only profoundly in its all classical branches but also widely from problems arising from other branches of science such as mathematical statistics and physics. The essential components of probability theory are experimental outcomes (called sample points), events, random variables, and stochastic processes. The latter two are mathematical abstractions of non-deterministic measured quantities that may either be a single value or evolve over time in a random fashion. If a random experiment is repeated many times, a sequence of random events will demonstrate certain patterns which can be studied and predicted. Two representative mathematical results describing such patterns are the law of large numbers and the central limit theorem, which have been crowned as being the first two of the three pearls of probability theory. (The law of iterated logarithm, the third pearl of probability theory, has not yet had as big an impact on applications that can be compared with the other two, since it cannot be observed even in a large number of replications of the experiment.) The laws of large numbers have become the stepping stone between probability theory and mathematical statistics. On one hand, the basic goal of probability theory is to calculate the probabilities of events under a given probabilistic model. On the other hand, mathematical statistics in a certain sense handles the inverse of the problems of probability theory. In other words, mathematical statistics prepares itself to clarify the structure of probabilistic-statistical models based on actual observations of various events. While it is difficult to forecast the general principles governing the behavior of a small set of random variables, the laws of large numbers encapsulate the notion that large sets of random variables tend to lose various aspects of their randomness. 8

9 They stabilize, patterns emerge, and their general behavior becomes fairly predictable. Indeed, the law of large numbers provides a rigorous mathematical description for the statistical laws abstracted from the empirical observation that the average of the results obtained from a large number of trials should be close to a fixed value (called the expected value or mean in probability theory and mathematical statistics), and will tend to become closer as more trials are performed. The first special form of the law of large numbers with rigorous mathematical proof was given by the prominent Swiss mathematician Jacob Bernoulli in his renowned book Ars Conjectandi (The Art of Conjecturing) published posthumously in His theorem is presently called the weak law of large numbers for Bernoulli trials. According to Bernoulli s theorem, if S n is the number of occurrences of an event A in n independent trials and p the constant probability of occurrence of event A in each of the independent trials, then for all positive real numbers ε, { } lim P S n n n p < ε = 1; that is, in probability terminology, S n /n converges to p in probability. This theorem was extended in the next 100 years or so by the great French mathematician and physicist Siméon D. Poisson and the eminent Russian mathematician Pafnuty L. Chebychev. It was Poisson who coined the phrase the law of large numbers (in French, la loi des grands nombres ). In 200 years or so after Bernoulli s weak law of large numbers, the French mathematician Émile Borel obtained the strong law of large numbers (SLLN) for Bernoulli trials, which concludes in probability terminology that S n /n converges to p almost certainly (a. c.); that is, { P lim n } S n n = p = 1. In 1933, the preeminent Soviet Russian mathematician Andrey N. Kolmogorov inaugurated the modern era in probability theory in his classic monograph Foundations 9

10 of the Theory of Probability. Kolmogorov there successfully gives probability theory a rigorous axiomatic basis, harnessing the full power of measure theory by regarding a probability event function as a measure of mass one defined on the σ-algebra of events. His SLLN declares that, for a sequence of i.i.d. random variables X 1, X 2,... and a real number µ, the following are equivalent: (i) The expected value of X 1 exists and is µ; that is, E X 1 < and EX 1 = µ. (ii) The sample mean converges to µ with probability one; that is, 1 n n X i µ a. c. i=1 This completes the line of work started by Jacob Bernoulli; i.e., the preceding Bernoulli s weak law of large numbers. Kolmogorov s SLLN is the precise form of the folklore idea of the law of averages, and shows convincingly that the Kolmogorov s axiomatic system has successfully captured the true essence of probability theory. Kolmogorov s SLLN was extended by Marcinkiewicz-Zygmund (1937) and Feller (1946) who proved SLLNs for i.i.d. random variables using more general norming sequences. The classic SLLNs can be extended in various directions and provides intuition for many other theories. Some SLLNs can be obtained under weakened assumptions, such as for random variables which are independent but not necessarily identically distributed, or for random variables which are pairwise independent (Chow and Teicher (1997, Section 5.2)). Some can hold in more general forms, for example, weighted sums of random variables. Stout (1974, Chapter 4) gives an excellent survey of known results up to 1974 on the SLLN problem for weighted sums of independent random variables. Martingale theory has SLLN type theorems derived via Kolmogorov s inequality (Feller (1971, Sections VII.8 and VII.9)). Ergodic theory, motivated by problems of statistical physics, has its foundations with the SLLN type theorems. The underlying idea is that for certain systems the time average of their properties converges to the average over the entire space (the so-called ensemble average). Two of the most important examples 10

11 are pointwise ergodic theorems of Birkhoff and von Neumann (Shiryaev (1996, Chapter V)). In mathematical statistics, the preceding SLLN type theorems provide numerous consistent estimators and statistics. 1.2 Strong Law of Large Numbers for Banach Space Valued Random Elements In the early 1950s, Probability in Banach Spaces, as a branch of modern mathematics, was initiated by the consideration of a stochastic process as a random element in a function space (a measurable function from a probability space to a function space) and, in particular, with the pioneering work by Fortet and Mourier (1953) on the law of large numbers and the central limit theorem for sums of independent identically distributed Banach space valued random variables (henceforth to be referred to as random elements). All technical definitions mentioned in Sections 1.2 and 1.3 will be reviewed in Chapter 2. The laws of large numbers for identically distributed (real-valued) random variables were extended to normed linear spaces by Mourier (1953) and Taylor (1972). Mourier (1953) established an analogue of the classical Kolmogorov s SLLN. Specifically, Mourier showed that, for a sequence of i.i.d. random elements {V n, n 1} in a real separable Banach space, if the expected value of V 1, denoted by EV 1, exists (the expected value of a random element is defined to be its Pettis integral), then 1 n n (V i EV 1 ) 0 a. c. i=1 Taylor (1972) provided conditions for identically distributed random elements in normed linear spaces to obey the weak law of large numbers. To obtain the corresponding results for the non-identically distributed random elements, additional conditions on the distributions of the random elements and/or on the Banach space itself are needed. A decisive step to the modern development of probability in Banach spaces was the introduction by Beck (1962) of a convexity condition on normed linear spaces equivalent to the validity of the extension of a 11

12 classical SLLN of Kolmogorov. Hoffmann-Jørgensen and Pisier (1976) established a SLLN by assuming the underlying real separable Banach space is of Rademacher type p (1 p 2). Actually, they showed that for a sequence of independent random elements {V n, n 1} with zero expected values in a real separable Banach space, the Banach space is of Rademacher type p (1 p 2) if and only if the following holds: E V n p < implies n p 1 n n V i 0 a. c. i=1 Thus Hoffmann-Jørgensen and Pisier (1976) provided an actual characterization of Rademacher type p (1 p 2) Banach spaces in terms of SLLN. A detailed discussion may be found in Taylor (1978, Chapter IV). The study of the SLLN for weighted sums of independent random variables contributes much to its extension to the SLLN for weighted sums of independent random elements. Adler and Rosalsky (1987a) and (1987b) presented a general SLLN for weighted sums of stochastically dominated random variables, which is general enough to include, as a special case, Feller s (1946) celebrated extension of the Marcinkiewicz-Zygmund SLLN (e.g., Chow and Teicher (1997, p. 125)). Adler and Rosalsky (1987a) did not require the summands to be independent. The hypotheses involve both the behavior of the tail of the distribution of the dominating random variable and the growth behavior of the weights and norming constants. Furthermore, the centering sequence is random. A result of Adler and Rosalsky (1987b) for weighted sums of i.i.d. random variables is substantially improved by Sung (1997) who obtained the same theorem but under less stringent conditions. Based on the work of Adler and Rosalsky (1987a) and (1987b), Adler, Rosalsky, and Taylor (1989) established a SLLN for weighted sums of independent random elements in normed linear spaces. The hypotheses involve the distributions of the independent random elements, the growth behaviors of the weights and norming constants, and for some of the results a geometric condition is imposed on the normed 12

13 linear space. Moreover, Adler, Rosalsky, and Taylor (1989) showed that Feller s (1946) famous result generalizing the Marcinkiewicz-Zygmund SLLN holds for random elements in a real separable Rademacher type p (1 < p 2) Banach space. Adler, Rosalsky, and Taylor (1992) extended the work of Adler, Rosalsky, and Taylor (1989) to the case of random weights. They also obtained a SLLN under a uniform integrability type condition instead of under the geometric condition that the space is of Rademacher type p (1 < p 2). Moreover, they established a SLLN for weighted sums of random elements in real separable semi-normed linear spaces which improves one of the earlier results of Adler, Rosalsky, and Taylor (1989). Furthermore, Adler, Rosalsky, and Taylor s (1989) extension to a Banach space setting of Feller s (1946) famous generalization of the Marcinkiewicz-Zygmund SLLN is obtained as a special case of a very general result of Cantrell and Rosalsky (2002). Therein, necessary and, separately, sufficient conditions are provided for a sequence of independent random elements to obey a SLLN. No conditions are imposed on the underlying Banach space for the necessity result, but for the sufficiency result, it is assumed that the Banach space is of Rademacher type p (1 p 2). Moreover, their necessity result extends to Banach space setting a result of Martikainen (1979) obtained for the random variable case and the sufficiency result also includes a well-known SLLN due to Heyde (1968) for the random variable case. Cantrell and Rosalsky (2004) established a SLLN for a sequence of independent random elements satisfying a uniform integrability type condition where no additional conditions are imposed on the underlying Banach space. Their main result includes as corollaries the SLLN of Adler, Rosalsky, and Taylor (1992) for a sequence of independent random elements satisfying a uniform integrability type condition and the SLLN of Taylor and Wei (1979) for a uniformly tight sequence of independent random elements. The laws of large numbers in Banach spaces provide powerful tools for many problems in stochastic process, decision theory, quality control, and statistical estimation 13

14 theory. Since some stochastic processes can be regarded as being a random element in particular function spaces, the laws of large numbers for random elements may be applied. In decision theory, the laws of large numbers can be applied to develop consistent statistical decision procedures. Quality control is an important industrial application of statistics. Consistent estimators of the parameters in a continuous production process can be constructed by using a law of large numbers for weighted sums of random elements. In the density estimation problem, the intuitive frequency histogram idea can be extended to a function space approach, and the laws of large numbers in Banach space can be applied under suitable conditions. A detailed discussion may be found in Taylor (1978, Chapter VIII). 1.3 Motivation and Organization of Dissertation Let {V n, n 1} be a sequence of random elements defined on a probability space (Ω, F, P) taking values in a real separable Banach space with norm. Suppose that EV n exists for all n 1. Let {a n, n 1} and {, n 1} be sequences of constants with 0 <. Then {a n (V n EV n )} is said to obey the general SLLN with norming constants {, n 1} if the normed weighted sum n i=1 a i(v i EV i )/ converges almost certainly to 0 (the identity of the Banach space as an abelian group under addition), and this will be written as This dissertation deals with two broad cases: n i=1 a i(v i EV i ) 0 a.c. (1.1) (i) (ii) obtaining SLLNs assuming the {V n, n 1} are independent and where we impose a condition on the underlying Banach space, obtaining SLLNs irrespective of the joint distributions of the {V n, n 1} and where no condition is imposed on the underlying Banach space. The work of part (i), which is established in Chapter 3, is inspired by Sung s (1997) extension for Theorem 2 of Adler and Rosalsky (1987b) (Proposition below). 14

15 Adler and Rosalsky (1987a) establish some SLLNs for weighted sums of random variables under rather general conditions. Therein, it is not assumed that the underlying random variables are independent or identically distributed or even integrable. Adler and Rosalsky (1987b) in their follow-up article provide sets of necessary and/or sufficient conditions for the SLLN to hold for weighted sums formed from sequences of independent and identically distributed (i.i.d.) random variables. In particular, Fernholz and Teicher s (1980) main theorem is a special case of Adler and Rosalsky s (1987b) Theorem 2 (Proposition below) taking a n = 1, = ϕ(d n ) and c n = EX n for n 1 where ϕ is a function defined for positive x such that ϕ(x)/x β is decreasing for some β > 1 and 0 < d n is a sequence of real numbers satisfying d n+1 /d n 1. Proposition (Theorem 2 of Adler and Rosalsky (1987b)). Let {X n, n 1} be a sequence of i.i.d. L q random variables for some 1 q < 2. Let {a n, n 1} and {, n 1} be sequences of constants with 0 <, ( ) a n 1 = O, (1.2) n 1/q and Then the SLLN holds. n a i = O( ). (1.3) i=1 n i=1 a i(x i EX i ) 0 a.c. (1.4) Sung s (1997) second result, which is stated in Proposition below, improves Theorem 2 of Adler and Rosalsky (1987b) (Proposition 1.3.1). Proposition (Theorem 2 of Sung (1997)). Let {X n, n 1} be a sequence of i.i.d. L q random variables for some 1 q < 2. Let {a n, n 1} and {, n 1} be sequences of constants with 0 <. Assume that condition (1.2) holds. Then (i) (1.4) holds if 1 < q < 2, (ii) (1.4) can fail if q = 1 and condition (1.3) fails. 15

16 Sung (1997) thus improved Theorem 2 of Adler and Rosalsky (1987b) (Proposition 1.3.1) by showing that condition (1.3) is not needed when 1 < q < 2, and that condition (1.3) is essential when q = 1. Sung (1997) established part (ii) of Theorem 2 of Sung (1997) (Proposition 1.3.2) via an example; that is, Sung (1997) gave an example showing that condition (1.3) cannot be dispensed with in Theorem 2 of Adler and Rosalsky (1987b) (Proposition 1.3.1) when q = 1. On the other hand, Theorem 6 of Adler, Rosalsky, and Taylor (1989), which is stated in Proposition below, extends Theorem 2 of Adler and Rosalsky (1987b) (Proposition 1.3.1) from the real line, which is of Rademacher type 2, to a Rademacher type p (1 < p 2) Banach space. Proposition (Theorem 6 of Adler, Rosalsky, and Taylor (1989)). Let 1 < p 2 and let {V n, n 1} a sequence of independent random elements in a real separable Rademacher type p Banach space X. Suppose that {V n, n 1} is stochastically dominated by a random element V in the sense that for some constant D <, P{ V n > t} DP{ DV > t}, t 0, n 1. Moreover, suppose that E V q < for some 1 < q < p. Let {a n, n 1} and {, n 1} be sequences of constants satisfying 0 < and conditions (1.2) and (1.3). Then the SLLN (1.1) holds. Therefore, we were motivated to extend part (i) of Sung s (1997) second theorem by obtaining an improved version of Theorem 6 of Adler, Rosalsky, and Taylor (1989). Indeed we obtained in Theorem 3.2.1, the main result in Chapter 3, an improved version of Theorem 1 of Adler, Rosalsky, and Taylor (1989) (Proposition in Chapter 3). Theorem readily provides in Theorem the desired improved version of Theorem 6 of Adler, Rosalsky, and Taylor (1989). In our main result, Theorem 3.2.1, we impose the crucial geometric condition on the real separable Banach space that it is of Rademacher type p (1 p 2). De 16

17 Acosta (1981) established in his Theorem 4.1 the following Marcinkiewicz-Zygmund type SLLN characterization for a real separable Banach space being of Rademacher type p (1 p < 2). The implication ((i) (ii)) was also obtained by Azlarov and Volodin (1981). Proposition (Theorem 4.1 of de Acosta (1981)). Let 1 p < 2. Let X be a real separable Banach space. Then the following are equivalent: (i) The Banach space X is of Rademacher type p. (ii) For every sequence of i.i.d. random elements {V n, n 1} in X with E V 1 p <, n i=1 (V i EV i ) n 1/p 0 a. c. (1.5) However, de Acosta (1981) did not provide an explicit example wherein the SLLN fails for a Banach space which is not of Rademacher type p. This motivated us to take advantage of Example 7.11 of Ledoux and Talagrand (1991, p. 190) (Example in Chapter 3). We also noted in Remark (iv) that in the special case where {V n, n 1} is a sequence of i.i.d. random elements with E V 1 q <, a n = 1, = n 1/q, n 1 where 1 < q < p 2 and the underlying Banach space is of Rademacher type p, Theorem reduces to the Marcinkiewicz-Zygmund type SLLN n i=1 (V i EV i ) n 1/q 0 a. c. (1.6) of Woyczyński (1980, Theorem 4.1). This result of Woyczyński (1980) of course does not follow from Theorem 6 of Adler, Rosalsky, and Taylor (1989) because condition (1.3) does not hold. By Theorem 4.1 of de Acosta (1981) (Proposition 1.3.4) or by the cited result of Azlarov and Volodin (1981), the Marcinkiewicz-Zygmund type SLLN (1.6) holds under the assumption that the Banach space X is of Rademacher type q, which is weaker than X being Rademacher type p. Indeed, Example shows explicitly that the Marcinkiewicz-Zygmund SLLN (1.6) can fail in a Banach space setting without the Rademacher type p hypothesis. Inspired by Beck (1963), we also construct in Example 17

18 3.2.3 a sequence of independent but not identically distributed random elements to serve the same purpose apropos of Theorem As was mentioned above, de Acosta (1981) provided a characterization in his Theorem 4.1 of Rademacher type p (1 p < 2) Banach spaces via a Marcinkiewicz- Zygmund type SLLN. The key result used by de Acosta (1981) to prove the SLLN (1.5) is the following result of de Acosta (1981, Theorem 3.1). Proposition (Theorem 3.1 of de Acosta (1981)). Let 1 p < 2 and let X be a real separable Banach space. Then for every sequence of i.i.d. random elements {V n, n 1} in X with E V 1 p <, the SLLN (1.5) holds if and only if n i=1 (V i EV i ) P 0. (1.7) n 1/p De Acosta (1981, Theorem 3.1) and de Acosta (1981, Theorem 4.1) together assert that Rademacher type p (1 p < 2) Banach spaces can be characterized by the Marcinkiewicz-Zygmund type weak law of large numbers (1.7). In summary, we establish in Chapter 3 the work of part (i) obtaining SLLNs assuming the {V n, n 1} are independent and the underlying Banach space is of Rademacher type p (1 p 2). Moreover, Theorem and Theorem are new results even when the underlying Banach space is the real line R. The work of part (ii), which is presented in Chapter 4, is a parallel development of the work of part (i) presented in Chapter 3 but the arguments are distinctly different. With the work of part (i) in hand, it seemed natural to develop a 0 < q < 1 version of Theorem Note that there are two cases for the real line version of the Marcinkiewicz-Zygmund SLLN: the condition for random variables being L q integrable for 1 q < 2 and for 0 < q < 1. Since Sawyer (1966), Chatterji (1970), and Martikainen and Petrov (1980) demonstrated that the real line version of the Marcinkiewicz-Zygmund SLLN holds without the independence hypothesis when 0 < q < 1, we were inspired to dispense with the independence assumption. We do not even impose further conditions, 18

19 such as a Rademacher type p (1 < p 2) condition, on the underlying Banach space. We obtain in Theorem 4.2.1, the first main result of Chapter 4, a SLLN of the form n i=1 a iv i 0 a.c. (1.8) which is parallel to the 0 < q < 1 part of the real line version of the Marcinkiewicz-Zygmund SLLN. We now recall the real line version Marcinkiewicz-Zygmund SLLN, which generalizes the classical Kolmogorov s SLLN as was mentioned in Section 1.1. Its proof may be found in Chow and Teicher (1997, p. 125). Proposition (Real line version Marcinkiewicz-Zygmund SLLN). Let {X n, n 1} be a sequence of i.i.d. random variables and let 0 < q < 2. Then n i=1 X i nc n 1/q 0 a. c. for some finite constant c if and only if E X 1 q <. In such a case, c = EX 1 if 1 q < 2 while c is arbitrary (and hence may be taken as zero) if 0 < q < 1. Theorem establishes a SLLN of the form (1.8) for a sequence of Banach space valued random elements {V n, n 1} which is stochastically dominated by a random element V with E V q < for some 0 < q < 1. The conclusion (1.8) holds irrespective of the joint distributions of the {V n, n 1}. The real line version of the Marcinkiewicz-Zygmund SLLN has a n 1 and = n 1/q, n 1. In Theorem we impose the condition ( ) a n 1 = O. (1.9) n 1/q which is of course automatic if a n 1 and = n 1/q, n 1. 19

20 Petrov (1973, Theorem 1), Rosalsky and Stoica (2010, Theorem 2.1), and Rosalsky and Stoica (2010, Theorem 2.2) obtained real line SLLNs of the form n i=1 X i 0 a. c. irrespective of the joint distributions of the random variables {X n, n 1}. We were thus enlightened to extend their results to the general form (1.8) for Banach space valued random elements. Petrov (1973, Theorem 1) motivated us to replace condition (1.9) by the condition ( ) q an < (1.10) under which we obtain a SLLN in Theorem 4.2.2, our second main result in Chapter 4. When a n /, (1.10) is stronger than (1.9). Though the 0 < q < 1 part of Theorem is thus a direct corollary of Theorem 4.2.1, Theorem extends Theorem in the sense that it can produce the SLLN (1.8) even when q = 1. Theorem is new even when the Banach space is the real line and as a corollary it yields Theorem 1 of Petrov(1973). Inspired by Rosalsky and Stoica (2010, Theorem 2.1), and Rosalsky and Stoica (2010, Theorem 2.2), we obtain the SLLN (1.8) under different conditions in Theorem and Theorem 4.2.4, respectively. Theorems and are also new even when the underlying Banach space is the real line, and they can incorporate as corollaries the results of Rosalsky and Stoica (2010, Theorem 2.1) and Rosalsky and Stoica (2010, Theorem 2.2), respectively. While the Banach space SLLN results obtained in the current work are new even in the random variable case, some corollaries or special cases of some of the random element (and random variable) results are well known as we have discussed above. Consequently, the current work is a bona fide extension of previously established results. Illustrative examples are provided throughout to compare the results or 20

21 show how the results improve upon or are different from other results in the literature. Examples are also provided to show that the results are sharp. Prior to the presentation of the main results in Chapters 3 and 4, notation, definitions, and some relevant results about Banach spaces are presented in Chapter 2, Section 1. Probabilistic concepts in Banach spaces are presented in Chapter 2, Section 2. Chapter 2, Section 3 contains the lemmas needed in Chapters 3 and 4. We end this chapter by mentioning that mean convergence versions of the Marcinkievicz-Zygmund SLLN have been investigated for both the cases of a sequence of random variables and a sequence of Banach space valued random elements. Klass (1973, Corollary 12) proved for a sequence of i.i.d. random variables {X n, n 1} with E X 1 p < for some p in [1, 2) that E n i=1 lim (X i EX i ) = 0. n n 1/p Korzeniowski (1984) extended this result of Klass (1973) to the case of a sequence of random elements taking values in a real separable Rademacher type q (1 < q 2) Banach space X (which is automatic if X = R). Specifically, it follows from Theorem 2 of Korzeniowski (1984) that for a sequence of i.i.d. random elements {V n, n 1} taking values in a real separable Banach space which is of Rademacher type q for some 1 < q 2, if E V 1 p < for some 1 p < q, then E n i=1 lim (V i EV i ) = 0. n n 1/p However, in this dissertation, we will concentrate on obtaining results for the a.c. convergence to 0 of normed partial sums and we will not obtain mean convergence results. 21

22 CHAPTER 2 PRELIMINARIES: DEFINITIONS, LEMMAS, AND NOTATION 2.1 Basic Concepts of Banach Spaces Some definitions, lemmas, and notation need to be presented prior to stating and proving the main results. A nonempty set X is said to be a (real) linear space if there is defined a binary operation of addition which makes X an abelian group and an operation of multiplication by (real) scalars which satisfy the distributive and identity laws; this is stated more precisely as follows. (a) To every pair of element (u, v) X X, there corresponds an element w X such that w = u + v. (b) To every u X and t R, there corresponds an element tu X. (c) The operations defined in (a) and (b) satisfy, for all u, v, w X and all s, t R, the following seven properties: (i) u + v = v + u, (ii) (u + v) + w = u + (v + w), (iii) u + v = u + w implies v = w, (iv) 1u = u, (v) (st)u = s(tu), (vi) (s + t)u = su + tu, (vii) s(u + v) = su + sv. The zero element of X is denoted by 0. While this is the same symbol as the real number 0, it should be clear from the context as to whether 0 refers to 0 X or 0 R. A real linear space X is said to be normed if there is a real-valued function defined on X and denoted by such that satisfies, for all u, v X and all t R, the following three properties: (i) u 0 and u = 0 if and only if u = 0, (ii) u + v u + v, (iii) tu = t u. 22

23 The function is then called a norm on X. Property (ii) above is called the triangle inequality. A sequence {v n, n 1} in a normed linear space X is said to converge to an element v of X if lim n v n v = 0. This will be denoted by lim n v n = v or by v n v as n. A sequence {v n, n 1} in a normed linear space X is said to be a Cauchy sequence if for every ε > 0, there exists an integer N such that v n v m < ε whenever n N and m N; i.e., lim sup v m v n = 0. n m>n A normed linear space X is said to be complete if every Cauchy sequence of X converges to an element of X. A complete normed linear space is called a Banach space. A subset S of a normed linear space X is said to be dense in X if its closure (that is, the smallest closed subset of X containing S) equals X. If X has a countable dense subset, then X is said to be separable. In the following examples we list several particular real Banach spaces. Example The space l p, 1 p <, is the class of all real sequences v = (v 1, v 2,...) such that k=1 v k p <. With the norm defined by ( ) 1/p v p = v k p, each of the spaces l p, 1 p <, is a real separable Banach space. k=1 Example The space l is the collection of all bounded real sequences v = (v 1, v 2,...). With the norm defined by v = sup{ v k, k 1}, l is a real Banach space which is not separable (e.g., Taylor (1978, p. 10)). Let c 0 denote the subspace of l which consists of the real sequences that converge to zero. With the same norm as l, c 0 is a real separable Banach space. 23

24 Example The space L p (R), 1 p <, is the class of all real Lebesgue measurable functions v( ) on R such that R v(t) p dt <. With the norm defined by ( 1/p v p = v(t) dt) p, R each of the spaces L p (R), 1 p <, is a real separable Banach space. Example The space L (R) is the class of all real Lebesgue measurable functions v( ) that are bounded almost everywhere (a.e.) on R with respect to Lebesgue measure. With the norm defined by v = inf{δ : v(t) δ a.e.}, the space L (R) is a Banach space which is not separable (e.g., Taylor (1978, p. 11)). The norm v is called the essential supremum of v( ) and is also denoted by β( v ). The collection of all continuous linear functionals (that is, continuous real-valued linear functions) defined on a normed linear space X is called the dual space of X and is denoted by X. We recall that a linear functional is a function f : X R satisfying f (au + bv) = af (u) + bf (v) for all u, v X and all a, b R. A sequence {, n 1} in a Banach space X is said to be a Schauder basis for X if for each v X there exists a unique sequence of scalars {t n, n 1} such that v = lim n n t k b k. When X has a Schauder basis {, n 1}, a sequence of linear functionals {f k, k 1} can be defined by k=1 f k (v) = t k, k = 1, 2,... where v X and v = lim n n k=1 t kb k. The linear functionals {f k, k 1} X are called the coordinate functionals. 24

25 The following Theorem is the Riesz Representation Theorem (e.g., Royden (1988, p. 132)) and it will be used in Example below. The Riesz Representation Theorem is a crowning achievement in twentieth century mathematics. Theorem (Riesz Representation Theorem). For each f in the dual space of L p (R), 1 p <, there exists g f L q (R) where 1/p + 1/q = 1 (q = if p = 1) such that f (h) = h(x)g f (x)dx for all h L p (R). R Remark The Riesz Representation Theorem is termed a representation theorem because it provides a concrete representation for the members of the dual space of L p (R), 1 p <. Informally, Theorem asserts that the dual space of L p (R), 1 p < is L q (R) where 1/p + 1/q = 1 (q = if p = 1). 2.2 Probability in Banach Spaces Let (Ω, F, P) be a probability space. Let X denote a real separable Banach space with a norm. Let X be equipped with its Borel σ-algebra B(X ); i.e., B(X ) is the σ-algebra generated by the class of open subsets of X determined by the metric d(u, v) = u v, u, v X. A random element V in X is a F-measurable transformation from Ω to the measurable space (X, B(X )); i.e., V 1 (A) F for all A B(X ). Remark A random element is a generalization of a random variable since the Borel σ-algebra generated by all intervals of real numbers of the form (, b) is the class of Borel subsets of R. Therefore, V is a random element in R if and only if V is a random variable. Furthermore, random elements in an n-dimensional Euclidean space R n are n-dimensional random vectors. The following Proposition shows that some properties of random variables can be extended to the setting of random elements. A further discussion may be found in Taylor (1978, Chapter II). 25

26 Proposition (Taylor (1978)). (i) Let V n, n 1 be a sequence of random elements in a Banach space X such that V n (ω) converges to V (ω) for each ω Ω. Then V is a random element in X. (ii) Let V be a random element in a Banach space X and let Y be a random variable. Then YV is a random element in X. (iii) If the real Banach space X is separable, then V W is a random variable whenever V and W are random elements in X. In particular, taking W = 0, V is a random variable if V is a random element. (iv) If the Banach space X is separable, then a function V : Ω X is a random element in X if and only if f (V ) is a random variable for each f X. Remark (i) The necessity half in Proposition (iv) is true without the assumption that X is separable. (ii) Not all of the properties of random variables can be extended to the setting of random elements. For example, the sum of two random variables are random variables, but the sum of two random elements in a Banach space X may not be measurable. However, if X is separable, then we see from Proposition (iv) that the sum of two random elements in X is a random element in X. (iii) Taylor (1978, p. 26) presented an example showing that if X is not separable, then V W is not necessarily a random variable where V, W, and V W are random elements in X. Consequently, Proposition (iii) can fail without the assumption that X is separable. We now define modes of convergence of a sequence of random elements in a real separable Banach space. Let {V n, n 1} be a sequence of random elements in a real separable Banach space X. Then {V n, n 1} converges to a random element V in X (i) { with probability one or almost certainly (a. c.) if P this is denoted V n V a. c. (or lim n V n = V a. c.). lim V n V = 0 n } = 1, and (ii) (iii) in probability if lim n P{ V n V ε} = 0 for all ε > 0, and this is denoted V n P V. in the rth mean for r > 0 if E V n r < for all n 1 and lim E V n V r = 0, and n L this denoted V r n V. Necessarily, we have E V r <. 26

27 A random element in a Banach space and the underlying probability measure induce a probability measure on the Banach space and its Borel subsets. The probability distribution of a random element V in a Banach space X is the induced measure, denoted by P V, on (X, B(X )); i.e., P V {B} = P{V B}, B B(X ). The random elements V and W in X are said to be identically distributed if P{V B} = P{W B} for all B B(X ). A family of random elements in X is said to be identically distributed if its every pair is identically distributed. A finite set of random elements {V 1,..., V n } in X is said to be independent if for every choice of B 1,..., B n B(X ), P{V 1 B 1,..., V n B n } = P{V 1 B 1 } P{V n B n }. A family of random elements in X is said to be independent if its every finite subset is independent. The expected value or mean of a random element V in a real separable Banach space X, denoted EV, is defined to be the Pettis integral provided it exists; i.e., V has the expected EV in X if for each f X, we have E[f (V )] = f (EV ) (2.1) where X is the dual space of X. Note that the left-hand side of (2.1) makes sense because of Proposition (iv) and also note that necessarily f (V ) is integrable for each f X. The Pettis integral was introduced by Pettis in 1938 (Pettis (1938)). A complete characterization of when the Pettis integral exists was provided by Brooks (1969). A further discussion and details regarding the properties of the Pettis integral may be found in Hille and Phillips (1985, pp ). 27

28 The expected value of random elements enjoys similar properties as does the expected value of random variables (Proposition below) and sometimes can be obtained as in the random variable case (Proposition and Example below). We illustrate the definition of the expected value of a random element with the following very simple example (Example 2.2.1). More involved examples (Examples and 2.2.3) are presented below. Example Let X be an L 1 random variable and let v X where X is an arbitrary real separable Banach space. Let V = Xv. Then the expected value EV of V exists and is given by EV = (EX )v. (This is of course precisely what one would expect to be the expected value of V.) Proof : By Proposition (ii), V = Xv is a random element in X since v can be regarded as a degenerate random element in X. Then f (V ) = f (Xv) = Xf (v) for all f X since X (ω) can be regarded as a real scalar for each ω Ω. Thus, (2.1) holds since E[f (V )] = E[Xf (v)] = f (v)ex = f ((EX )v) for all f X. Therefore, the expected value EV of V exists and is given by EV = (EX )v. Proposition (Taylor (1978)). Let V, V 1 and V 2 be random elements in a real separable Banach space X, then (i) If EV 1 and EV 2 exist, then E(V 1 + V 2 ) exists and E(V 1 + V 2 ) = EV 1 + EV 2. (ii) If EV exists and t R, then E(tV ) exists and E(tV ) = tev. (iii) If E V <, then the expected value EV of V exists and EV E V. Proposition If V is a countably-valued random element in X taking values {v i, i 1}, then the expected value EV of V exists and is given by EV = provided v i P{V = v i } <. i=1 v i P{V = v i } i=1 28

29 Proof : Let v = v i P{V = v i }. Then v X since X is complete. Moreover, (2.1) i=1 holds since for each f X, E[f (V )] = f (v i )P{V = v i } i=1 = lim n = lim =f =f n f ( n f (v i )P{V = v i } i=1 lim n ( n ) v i P{V = v i } i=1 ) n v i P{V = v i } i=1 ( ) v i P{V = v i } i=1 (since f is continuous) =f (v). Thus, the expected value of V exists and is given by EV = v = v i P{V = v i }. Example If a Banach space X has a Schauder basis {, n 1} with coordinate functionals {f n, n 1}, then each random element V in X can be expressed as V = f n (V ) pointwise in ω Ω. If V has expected value EV X, then E[f n (V )] = f n (EV ) since each f n is in X. Thus i=1 EV = f n (EV ) = E[f n (V )]. (2.2) Note that the spaces l p, 1 p < share the same Schauder basis {v (n), n 1} where v (n) is the element of l p having 1 in its nth position and 0 elsewhere. Thus, each random element V in l p, 1 p < can be expressed as a sequence of random variables {f n (V ), n 1}; i.e., V = (f 1 (V ), f 2 (V ),...) = (V 1, V 2,...) (say). Furthermore, if 29

30 the expected value EV of V exists, then by (2.2) we get EV = E[f n (V )]v (n) = (E(f 1 (V )), E(f 2 (V )),...) = (EV 1, EV 2,...). (2.3) (Again, this is precisely what one would expect to be the expected value of V.) Paralleling the Riesz Representation Theorem which concerns the real separable Banach space L p (R), 1 p <, the following representation theorem for l p, 1 p < (Wilansky (1964, p. 91)) will be used in Remark below which pertains to Example Theorem For each f l p, 1 p <, there exists b(f ) = (b 1 (f ), b 2 (f ),...) l q where 1/p + 1/q = 1 (q = if p = 1) such that f (a) = a n (f ) for all a = (a 1, a 2,...) l p. Remark Let V = (V 1, V 2,...) be a random element in l p (1 p < ) as in Example If we assume E V n p < (2.4) (that is, (E V 1 p, E V 2 p,...) l 1 ), then we also obtain the expected value of V with the form (2.3) via Theorem as follows. Note at the outset that (2.4) implies that V n is integrable for each n 1. Let v = (EV 1, EV 2,...). Then v l p since ( ) 1/p v p = EV n p ( ) 1/p E V n p (by Jensen s Inequality) < (by (2.4)). By Theorem 2.2.1, f (V ) = V n (f ) for each f l p where b(f ) = (b 1 (f ), b 2 (f )),...) l q and 1/p + 1/q = 1 (q = if p = 1). 30

31 Then, for each m 1, m m V n (f ) V n (f ) V n (f ) V p b(f ) q (by Hölder s Inequality). Moreover, V p b(f ) q is integrable since E( V p b(f ) q ) = b(f ) q E V p ( ) 1/p = b(f ) q E V n p ( )] 1/p b(f ) q [E V n p [ ] 1/p = b(f ) q E V n p < (by (2.4)). (by Jensen s Inequality) (by Lemma 2.3.6) Thus, by the Lebesgue Dominated Convergence Theorem, ( ) E[f (V )] = E V n (f ) = E ( lim m = lim m E ) m V n (f ) ( m ) V n (f ) ( m = lim (EV n ) (f ) m = (EV n ) (f ) ) = f (v) (by Theorem 2.2.1) 31

32 recalling that v = (EV 1, EV 2,...). Hence, the expected value EV of V exists and is given by (2.3). Example Let V be a random element in X = L p (R), 1 p < (Example 2.1.3) with V Ω R (ω)(x) p dxdp(ω) <. Then the expected value of EV of V exists and is given by EV = V dp viewed as a function of x R; i.e., EV : R R is given Ω by x Ω V (ω)(x)dp(ω). (Once again, this is precisely what one would expect to be the expected value of V.) Figure 2-1 below is provided to help clarify the notion of the expected value of a random element V in L p (R), 1 p <. Figure 2-1. Expected Value of a Random Element in L p (R), 1 p < Proof : For fixed x R, V ( ) (x) is a random variable. Define a function v( ) on R by v(x) = EV ( ) (x) = V (ω) (x)dp(ω) for all x R. Ω Then v( ) is Lebesgue measurable. Since 1 p <, we have for each x R that v(x) p = EV ( ) (x) p E V ( ) (x) p by Jensen s inequality. So v( ) L p (R) since v(x) p dx E V ( ) (x) p dx R R 32

33 = = R Ω <. Ω R V (ω) (x) p dp(ω)dx V (ω) (x) p dxdp(ω) On the other hand, for fixed ω Ω, V (ω) ( ) L p (R). By the Riesz Representation Theorem (Theorem 2.1.1), for each f in the dual space of L p (R), there exists g f L q (R) where 1/p + 1/q = 1 (q = if p = 1) such that f (h) = h(x)g f (x)dx R for all h L p (R). Therefore, for each f in the dual space of L p (R), by first taking h( ) = V (ω) ( ) and then taking h( ) = v( ), we get E[f (V )] = = = = Ω Ω R R = f (v). f (V (ω) ( ))dp(ω) ( ) V (ω) (x)g f (x)dx dp(ω) R ( ) V (ω) (x)dp(ω) g f (x)dx Ω v(x)g f (x)dx (by Fubini s Theorem) Hence, the expected value EV of V exists and is given by EV = v = V dp. Ω The following example shows that the expected value EV can exist even if E V =. Example (Taylor (1978, p. 41)). For the real separable Banach space l 2, define a random element V such that V = nv (n) with probability c/n 2 where v (n) is the element of l 2 having 1 in its nth position and 0 elsewhere and c is an appropriate constant. Note that E V 2 = n c n = c 1 2 n =, 33

34 However, by Proposition 2.2.3, EV = nv (n) P(V = nv (n) ) = nv (n) c ( c n = 2 1, c 2,..., c ) n,... l 2. Let {ε n, n 1} be a symmetric Bernoulli sequence; i.e., {ε n, n 1} is a sequence of independent and identically distributed (i.i.d.) random variables with P{ε n = 1} = P{ε n = 1} = 1/2, n 1. A symmetric Bernoulli sequence is also referred to as a Rademacher sequence. Let X = X X X, and define { C(X ) = (v 1, v 2,...) X : } ε n v n converges in probability. Let 1 p 2. Then a real separable Banach space X is said to be of Rademacher type p if there exists a constant 0 < C < such that p E ε n v n C v n p for all (v 1, v 2,...) C(X ). Hoffmann-Jørgensen and Pisier (1976) proved for 1 p 2 that a real separable Banach space is of Rademacher type p if and only if there exists a constant 0 < C < such that n p E V i C i=1 n E V i p i=1 for every finite collection {V 1,..., V n } of independent random elements in X with zero expected values. If a real separable Banach space is of Rademacher type p for some 1 < p 2, then it is of Rademacher type q for all 1 q < p. Every real separable Banach space is of Rademacher type (at least) 1 while the L p -spaces and l p -spaces are of Rademacher type min{2, p} for p 1. Every real separable Hilbert space and real separable finite-dimensional Banach space is of Rademacher type 2; in particular, the 34

35 real line R is of Rademacher type 2. The real separable Banach space c 0 (Example 2.1.2) is not of Rademacher type p for any p (1, 2] and for q [1, 2), the real separable Banach spaces L q and l q are not of Rademacher type p for any p (q, 2]. A detailed discussion of the above can be found in Chapter 9 of Ledoux and Talagrand (1991). The real Banach space l is not even separable as was mentioned in Example Useful Lemmas The classical and celebrated real line version of Lévy s Theorem (e.g., Chow and Teicher (1997, p. 72)), which asserts that the partial sums from a sequence of independent random variables converge almost certainly to a random variable S if and only if they converge in probability to S, has been extended to a real separable Banach space setting by Itô and Nisio (1968) and is stated as follows. Lemma (Itô and Nisio (1968)). Let {V n, n 1} be a sequence of independent random elements in a real separable Banach space X and set S n = n V i, n 1. i=1 Then S n converges a.c. to a random element S in X if and only if S P n S. Remark It follows from Lemma that in the definition of C(X ), the condition ε n v n converges in probability is equivalent to the condition ε n v n converges a.c. Now we introduce the notion of regular variation which has been proved fruitful in an increasing number of applications in probability theory (Feller (1971, VIII.8 and VIII.9) for a detailed discussion). A positive Borel function L defined on [0, ) is said to vary slowly 35

36 (at infinity) (or be slowly varying (at infinity)) if L(cx) lim x L(x) = 1 for all c > 0. A positive Borel function R on [0, ) is said to vary regularly (or be regularly varying) with exponent ρ ( < ρ < ) if it is of the form R(x) = x ρ L(x) with L slowly varying; i.e., R(cx) lim x R(x) = c ρ for all c > 0. Clearly, a function is slowly varying if and only if it is regularly varying with exponent ρ = 0, and a positive Borel function L is slowly varying if and only if 1/L is slowly varying. For example, all powers of log x are slowly varying. Similarly, a function approaching a positive finite limit is slowly varying. Feller (1971) introduced the following two abbreviations: Z u (x) = x 0 y u Z(y)dy, Z u (x) = x y u Z(y)dy, < u <, (2.5) where Z is a regularly varying function, and explored their asymptotic properties as x (Lemmas and below). We will apply these properties in Examples 3.2.6, 3.2.8, 3.2.9, and Lemma (Feller (1971, p. 280)). Let Z > 0 vary slowly. Then the integrals in (2.5) converge at for u < 1 and diverge for u > 1. If u 1, then Z u varies regularly with exponent u + 1. If u < 1, then Z u varies regularly with exponent u + 1, and this remains true for u = 1 if Z 1 <. Lemma (Feller (1971, p. 281)). (i) If Z varies regularly with exponent ρ and Zu <, then u + ρ and where λ = (u + ρ + 1) 0. x u+1 Z(x) lim x Zu (x) = λ 36