Research on Independence of. Random Variables

Transcription

1 Applied Mathematical Sciences, Vol., 08, no. 3, - 7 HIKARI Ltd, Research on Independence of Random Variables Jian Wang and Qiuli Dong School of Mathematics and Statistics Shandong University of Technology, Zibo, China Copyright 08 Jian Wang and Qiuli Dong. This article is distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Abstract Taking two-dimensional random variable as the research object, we dig the profound connotation of independence and reveal the true nature of it. Then, from the point of view of entropy in information theory, we explore the relationship between each component of two-dimensional random variable and the equivalence between independence and non-correlation of them. Keywords: independence, generalized correlation, two-dimensional normal distribution, entropy Introduction Independence of random variables is an important part of probability theory. For example, the independence of samples is the condition of many fundamental theories in the mathematical statistics. In this paper, we do a research on independence of random variables. Research on Independence of Two-Dimensional Random Variable Definition.. Let X and Y be random variables taking values in the probability

2 Jian Wang and Qiuli Dong space,,p.let and from R, Β into itself be the real measurable functions which are the non-degenerate random variables. The covariance of and is denoted by the symbol Cov f X, g Y. We Cov f X, g Y call ρ = the correlation coefficient of f ( X ) and g( Y). D f X D g Y Then we say that and g Y are generalized correlate(resp. generalized non-correlate) if ρ 0 (resp. ρ = 0 ) holds. In particular, it is called a complete generalized correlation when ρ = holds. [6] Generalized correlation reflects the relationship among all real measurable functions defined on random variables X and Y. Then we will use this definition to explore the true nature of independence. Theorem.. Let and Y be the random variables. and g Y are the real measurable functions mentioned in definition.,then we have the equivalence where, is the joint distribution function of, and F x (resp. F y ) the distribution function of X (resp. Y ). Y Proof." " We know the condition ρ = 0,then we have Let (,,P) Assume ( Ω F ) f ( X ) g( Y ) f ( X ) g( Y ) ( ) ( ) f ( X ) X f ( X ) F xy = F xf y ρ = 0, X Y F( xy ) ( XY ) Ω F be a probability space with any set A F and B F. = E f X g Y E f X E g Y = A = f X I X g Y IB Y X ()

3 Research on independence of random variables 3 I ( X ) where (resp. I Y ) denotes the indicator function on the set A (resp. A B ).Then we have Clearly, F xy = F xf y. B = = E f( X) g Y E IA X IB Y P AB = = E f X E g Y E I X E I Y P A P B A B, X Y " " It can be proved that if X and Y are independent, then all real measurable functions defined on them are independent of each other. Clearly, ρ = 0. From the proof of Theorem., we can reveal the true nature of independence between random variables. That is, the independence of random variables means that occurrence of A dose not change the probability that B also occurs if X and Y are defined on A and B, respectively.based on the theory in this section, a distinction on independence and functional relationship of random variables should be made.[] We just know that two random variables are not independent if they are relevant.next,we will explore the true nature of this conclusion through lemma. Lemma. The random variables are not independent if they are relevant. Proof. Let X and Y be the random variables defined on the probability space (,,P) Ω F.We know that X and Y are relevant, then we have [, ] E[ X] E[ Y] E X Y For any AB, F,we assume where (resp. I Y ) denote the indicator on the set A (resp. B ).Then we have I ( X ) A B X = IA X Y = IB Y [ ] = = E XY E I A X I B Y P AB [ ] [ ] = = E X E Y E I X E I Y P A P B A B

4 4 Jian Wang and Qiuli Dong Clearly, P AB P A P B, that is, X and Y are not independent. Research on Independent and Irrelevant Problems of Normal Distribution Before we explore the question, we make a distinction as the Remark follows. Remark. The joint distribution of two normal distribution random variables don't necessarily follow the normal distribution problem. Random variables are independent of each other, certainly not relevant; the other hand, if two random variables are irrelevant, we can't prove that the two are independent of each other. Definition 3. Let fx ( x).then we call X be a continuous random variable with the density function H X fx x log fx x dx = () the entropy of the random variable X. Similarly, is called the joint entropy of XY,. Theorem 3.. Let ( XY, ) (, ) (, ) log (, ) H X Y = f x y f x y dxdy (3) be two-dimensional normal random variable. Then the relation of X and Y is either relevant or there is no functional relation between f ( X ) the real measurable function and g Y defined on them, respectively. Proof. Random variables X and Y are either relevant or irrelevant. We know that they are not independent if they are relevant. Then we have f ( X ) ρ = Clearly, and g Y defined on them have no relation. Theorem 3. shows that each component of two-dimensional normal random variable are either relevant or independent. 0

5 Research on independence of random variables 5 Definition 3.. Let ( XY, ) be the two-dimensional random variable with the density function pxy,. µ (resp. µ ) is the mean of X (resp. Y ) and σ(resp. X µ Y µ σ ) the standard deviation of X (resp. Y ).Put X = ; Y =,then we σ σ call X, Y the standardized random variable of XY, with density function f ( xy, ).we know Then we have Hence, H XY, H X, Y log σσ. We call H X, Y the standard entropy of XY,.[4] Under Definition 3., by example, we will explore the relationship between independence and entropy of two-dimensional normal distribution random variable. Example. Let distribution (,,,,0) (, ) = σσ ( σ + µ, σ + µ ) f xy p x y (, ) = (, ) log (, ) H X Y f x y f x y dxdy x+ = s y+ = t = σσ p σ x + µ, σ y + µ log σσ p σ x + µ, σ y + µ dxdy σ µ σ µ (, ) log σσ (, ) log (, ) H( XY) = p s t dsdt p s t p s t dsdt = log σσ +, = + ( XY, ) be the two-dimensional random variable with normal X µ Y µ N µ µ σ σ. Put X = ; Y =.Suppose that σ σ pxy (, ) is the density function of ( X, Y ),then we have (, ) (, ) log H XY = H X Y + σσ (4)

6 6 Jian Wang and Qiuli Dong where (, ) = (, ) log (, ) H X Y f x y f x y dxdy Hence, ( ) =log + f x, y logexp x ρxy + y dxdy π ρ ρ =log π ρ π ρ (, ) = log + logσσ H XY It is obvious that when the mean and variance are known, the smaller the correlation coefficient is, the larger the entropy H XY,.And when ρ = 0 ρ H( XY ) holds, we have the maximum, and H X, Y = H X + H Y holds. Hence, X and Y are independent of each other. From the point of entropy, this example reveal that the independence and non-correlation among the components of two-dimensional normal random variables are equivalent. References [] He Chang, Entropy and variance, Journal of Kaifeng Institute of Education, (00), [] Junshan Xie, Weibing Deng, A distinction on independence and functional relationship of random variables, New Curriculum, [3] Nader Ebrahimi, Esfandiar Maasoumi, Ehsan S. Soofi, Ordering univariate distributions by entropy and variance, Journal of Econometrics, 90 (999), [4] Xingsi Li, Yuxi Jiang, Shaohua Pan, Entropy and optimization methods, Dalian University of Technology, 06.

7 Research on independence of random variables 7 [5] Yejiao Shi, Xiaotao Meng, Mutual relationship and classification between random variables, Journal of Shenyang Normal University, 3 (04), -5. [6] Zuoren Wang, Guifen Ye, Correlation of Random Variables, Statistics & Information Forum, 997. Received: August 7, 08; Published: September 8, 08