Statistics Assignment 2 HET551 Design and Development Project 1

Transcription

1 Statistics Assignment HET Design and Development Project Michael Allwright Haddon O Neill 7396 Monday, 3 June

2 Simple Stochastic Processes Mean, Variance and Covariance Derivation The following sections show the derivation of various parameters of the data set Y n Equation This data set is defined in y n = { n i= X i n =,, n = () Mean Derivation m(n) = E Y n n m(n) = E X i i= m(n) = n E X i m(n) = nµ Variation Derivation V (n) = Var (Y n ) ( n ) V (n) = Var X i Since the variance of sum is the sum of variances for a random variable, this expression can be rewritten as: i= V (n) = n var (X i ) i= V (n) = nσ 3 Covariance Derivation L cov(y L, Y K ) = Cov X i, i= K j= X j cov(y L, Y K ) = L i= K Cov (X i, X j ) j=

3 Representing this expression in a matrix it can be shown that across the diagonal we have the covariance of a single random variable with itself, and outside the diagonal we the covariance of two indepent random variables Cov (X, X ) Cov (X, X ) Cov (X, X K ) Cov (X, X ) Cov (X, X ) Cov (X, X K ) Cov (X L, X ) Cov (X L, X ) Cov (X L, X K ) j K Using the basic rules of Covariance, Cov (X, X) = Var (X), and Cov (X, Y ) = (for X and Y being indepent random variables), the matrix can be simplified as follows Var (X ) Var (X ) Var ( ) X L K Since Var (X) = σ the cov(y K, Y L ) = Kσ given that K < L i L Covariance Simulation of the Binomial Distribution Process Simulation for the case of X i being binomially distributed To simulate this process, the MATLAB script in Listing was to generate a few trajectories These trajectories are shown in Figure few = ; % Few being the amount of trajectories plotted p =, n =; % Graph variables defined colors = b g r c k ; figure (), clf (), axis ( 4), xlabel ( n Trials ), ylabel ( Trial success ); hold for on; i =:: few Xi = binornd (,p,,n ); Y =; Y= cumsum (Xi ); stairs (Y, sprintf ( %s,colors (i ))); hold off ; title ( sprintf ( Number of successes for Probability of %3 f over -% i trials,p, n )); saveas (gcf, sprintf ( p =%3 f n=%i pdf,p,n )); Listing : MATLAB Source Code to plot of several trajectories of the counting process Estimation of the Covariance of two variables in the sequence Y n through Simulation To estimate the covariance through simulation, a MATLAB script which utilized, trials was executed This script is shown in Listings, this script relies on the sample covariance function defined in Listings 3 The script returned the value 937 for the values indexed at and 8 in Y n (arbitrarily chosen) 3

4 4 3 3 Trial success n Trials Figure : Number of successes for Probability of over - trials From Section 3 the analytical expression for Covariance is: cov(y K, Y L ) = Kσ given that K < L For the Bernoulli case is σ = p( p), substituting this into the expression yields cov(y K, Y L ) = Kσ = Kp( p) which for K = 49 and L = 8, results in the analytical calculation of 987, this approximately matches the result from simulation 3 Calculation of the Joint Probability of the Expression P (X 3 =, X 4 = 7) The probability of S n = y occurring given that S n = y is given by ( ) ( n n P (S n = y, S n = y ) = n y y y ) p y ( p) n y For our chosen probability, that of p = P (Y 3 =, Y 4 = 7) = ( ) ( 3 ) 7 ( ) Using the MATLAB script in Listings 4, this analytical result was confirmed by simulation which gave the result P (Y 3 =, Y 4 = 7) 96 3 Stochastic Process of a Uniformly Distributed Variable From Equation we know that y n is a cumulative summation of the randomly distributed variable X i To determine the function in Equation, we first analytically determine the mean and variance of y(n) ỹ t (n) = y(n) m(n) v(n) () 4

5 % Defining variables p =, n =, trials = ; Z = zeros ( trials,(n +)); acc = ; for i =:: trials Xi = binornd (,p,,n ); % Inserts N ()= Y =; Y= cumsum (Xi ); % Returns if (Y (3)== && Y (4)==7) acc = acc +; Z(i,:)= Y; % Using cov as Sample Variance % Cov (Yk, Yl) where k < l k =49; l =8; SampleCov (Z(:,k+),Z(:,l +)) % Comparing to Sample Variance cov (Z(:,k+),Z(:,l +)) Listing : MATLAB Source Code to determine the covariance of two variables in the counting process function covariance = SampleCov (x,y) acc =; for i = :: size (x,) acc = acc +(( x(i)- mean (x ))*( y(i)- mean (y ))); covariance = acc /( size (x,) -); Listing 3: MATLAB Source Code to Calculate the Sample Covariance 3 Mean Calculation for the Random Variable X i Uniform(, ) The Probability Density Function (pdf) of a uniform random variable in the range, is easily defined It is simply a rectangular area which covers the range, with a height of, This ensures that the area under the pdf is unity f(x) =, x otherwise The expectation of X is then defined as: Substituting in the derived density function: ˆ EX = x f(x) dx EX = ˆ x dx = 4 x = (3)

6 % Finding P( X_n = y, X_n = y) %P( X_n = y, X_n = y) via calculation % where n =3, y =, n =4, y =7; P = nchoosek ((n -n ),(y -y ))* nchoosek (n,y )*p^y *( -p)^( n -y) % Defining variables p =, n =, trials =, Z= zeros ( trials,(n+)), acc =; for i =:: trials Xi = binornd (,p,,n ); Y =; Y= cumsum (Xi ); if (Y (3)== && Y (4)==7) acc = acc +; Z(i,:)= Y; % Results from Simulation, ProbSim = acc / trials Listing 4: MATLAB code to prove probability through simulation 3 Variance Calculation for the Random Variable X i Uniform(, ) Using the result in Equation 3, the Variance can now be defined such that: V ar(x i ) = ˆ V ar(x i ) = ˆ ( x x + ) dx = (x EX) f(x) dx V ar(x i ) = 3 = x3 x + x = ( ) Distribution of the Stochastic Process The transformation of the random variable shown in Equation can now be written numerically given X i Uniform(, ) ỹ t (n) = (y(n) ) 3 (4) As shown in Figure, as n increases there is convergence towards a limiting distribution From Figure, It appears that this distribution is Gaussian in shape, which is expected due to central limit theorem The central limit theorem states that the summation of indepent random variables leads to a Gaussian distribution 6

7 8 Frequency 6 4 n Yt(n) bins 3 4 Figure : 3D Histogram showing change in distribution as n increases The Multi-Dimensional Normal Distribution Probability Density Function for the Bi-variate Normal Distribution The Probability Density Function (pdf) for the Bi-variate Normal Distribution is as follows: f(x, y) = π e ρ σ σ ( ρ ) x σ xyρ + y σ σ σ () Calculating the Correlation Coefficient The expression for the covariance is shown in Equation 6 To determine the covariance of the pdf in Equation, The mean values µ x and µ y are set to zero and the pdf is substituted for the f(x, y) term Cov(x, y) = Cov(x, y) = ˆ ˆ ˆ ˆ (x µ x ) (y µ y ) f(x, y) dx dy (6) xy π e ρ σ x σ y ( ρ ) x σx xyρ σxσy + y σy dx dy (7) To avoid any mistakes in evaluating this integral, this expression was evaluated and simplified using Wolfram s Mathematica Software Package To get a clean solution to this integral, some logical assumptions were made about the variables ρ, σ x and σ y these are as follows: > ρ > 7

8 σ x > σ y > Given these assumptions, the covariance of the pdf in Equation is defined by the expression in Equation 8 Cov(x, y) = ρσ x σ y (8) Comparing the result in Equation 8 with the relationship for the covariance and correlation coefficient in Equation 9, it shown that ρ is the correlation coefficient Corr(X, Y ) = Cov(X, Y ) σ x σ y (9) 3 Proof that the Bivariate Normal Distribution is a special case of the Multivariate Normal Distribution The definition of the pdf for a multivariate joint Gaussian distribution is defined by: Where f X (x) f X, X,X n = exp{ (x m)t K (x m)} (π) n/ K / x = x x x n, m = m m m n = EX EX EX n, K = Var (X ) Cov (X, X ) Cov (X, X n ) Cov (X, X ) Var (X ) Cov (X, X n ) Cov (X n, X ) Cov (X n, X ) Var (X n ) Substituting X and Y into the covariance matrix for n =, K = Var (X) Cov (X, Y ) Cov (Y, X) Var (Y ) K = σ ρσ σ ρσ σ σ The inverse of the covariance matrix is defined as follows K = σ σ ( ρ ) σ ρσ σ ρσ σ σ Substituting this back into the joint Gaussian multivariate pdf, where the term ϕ is the exponent, in the term e ϕ, such that: 8

9 z frequency 3 y bins y x bins (a) 3D Histogram of the Rayleigh Distribution x (b) pdf of the Bivariate Normal Distribution Figure 3: Comparison of the Gaussian and Rayleigh Distributions σ ρσ σ ρσ σ σ σ (x m ) ρσ σ (y m ) ρσ σ (x m ) + σ (y m ) ϕ= (x m, y m ) σ σ ( ρ ) (x m, y m ) ϕ= σ σ ( ρ ) ϕ= x m σ ) (y m ) ρ (x m + σ σ ( ρ ) y m σ x m y m () Them and m are the centering points for the X, Y and n = normal distribution, setting the centering pointsm and m to zero standardizes the expression in Equation giving the standard bivariate normal distribution s exponent from the pdf ϕ= 4 x σ ρxy σ σ + y σ ( ρ ) Rayleigh Distribution The Rayleigh method generates indepent pairs of normally distributed variables The MATLAB code in Listing, generates these values and plots a 3D histogram showing the distribution for the random process For comparison, the code also generates a surface plot of the Probability Density Function for the Bivariate Normal Distribution The plots for the Rayleigh Histogram and Bivariate Normal pdf surface are shown in Figures 3a and 3b respectively As can be from the plots in Figures 3a and 3b, Both of these plots appear to share the Gaussian shape in both dimensions, centered around the same mean 9

10 function myrayleigh ( sigma, numvals ) nqz = e - ; % not quite zero constant angle = random ( unif,, * pi,, numvals ); radius = sigma * sqrt ( - * log ( random ( unif,nqz,,, numvals ))); x y = polcart ( angle, radius ); figure (), clf (), hist3 (x y, 6 6); xlabel ( x- bins ), ylabel ( y- bins ), zlabel ( frequency ); set (gcf, rerer, opengl ); set ( get (gca, child ), FaceColor, interp, CDataMode, auto ); pause ; mx = ; Cx = ; ; x = -::; for i =: length (x), for j =: length (x), f(i,j )=(/(* pi*det (Cx )^/))* exp (( -/)* (x(i) x(j) -mx )* inv (Cx )*( x(i);x(j) - mx )); figure (), clf (), surf (x,x,f); xlabel ( x ), ylabel ( y ), zlabel ( z ); Listing : MATLAB Source Code to generate a histogram of the Rayleigh Distribution and Standard Bivariate Normal Plane for Comparison