Appendix A Math Reviews 03Jan007 Objectives. Review tools that are needed for studying models for CLDVs.. Get you used to the notation that will be used. Readings. Read this appendix before class.. Pay special attention to the results marked with a *. 3. Review any other algebra text as needed. A. From Simple to Complex I With a simple equation: x = y I Or a complex equation: y = b 0 + b x + b x + + u I The same rules apply. Don t confuse messy and complex with hard and incomprehensible! i
ii APPENDIX A. MATH REVIEWS 03JAN007 A. Basic Rules Distributive law a (b + c) = (a b) + (a c) (A.) 4 ( + 3) = (4 ) + (4 3) ( ) ( 0 + x + x ) = ( ) (A.) = = ( 0 + x + x ) ( 0 + x + x ) = [ 0 + x + x ] [ 0 + x + x ] Multiplying by a b = a b = k k a b = ka kb (A.3) A.3 Solving Equations 3 = 3 = 4 4 3 = 4 4 3 = 8 = 3 Let p be the probability of an event, and = p p the odds (Note: = ex ). You should be able to work this derivation from to p and from p to without looking. = p p 9 = :9 : ( p) = p 9 ( :9) = :9 p = p 9 9 (:9) = :9 = p + p 9 = 9 (:9) + :9 Therefore, p = = p ( + ) 9 = :9 ( + 9) + = p 9 + 9 = :9 + = ex + e x :
A.4. EXPONENTS AND RADICALS iii A.4 Exponents and Radicals Zero exponent a 0 = (A.4) 3 0 = :788 0 = = e 0 Integer exponent a k = a (k) a; where (k) means repeat k times (A.5) 3 = = 8 e 3 = :788 :788 :788 = 0:086 Negative integer exponent a k = 3 = a (k) a = a k = 8 (A.6) Base e e = :7888846 : : : is a useful base. Notation is: e x or exp(x). e 0 = e = : 78 e = e e = 7:389 e 3 = e e e = 0:086 * Product of powers: multiplying as the sum of powers * Quotient of powers a M a N = [a (M + N) a] = a M+N (A.7) 3 4 = ( ) ( ) = 3+4 = 7 e 3 e 4 = (e e e) (e e e e) = e 3+4 = e 7 (A.8) a M [a (M) a] = an [a (N) a] = am N (A.9) e 5 = e e e e e e 3 e e e = e 5 3 = e
iv APPENDIX A. MATH REVIEWS 03JAN007 Power of powers (a M ) N = a MN (A.0) e 5 A.5 ** Natural Logarithms = (e e) (e e) (e e) (e e) (e e) = e 0 = e 5 Natural logarithms and exponentials are used extensively in statistics. A key reason is that they turn multiplication into addition. Here s why:. Every positive real number m can be written as m = e p. Example: Let m = 3. Find p such that e p = 3: (a) e = 7:389 : : : and e 3 = 0:086 : : : ) < p < 3. (b) e :5 = :8 : : : and e :6 = 3:464 : : : ) :5 < p < :6. (c) And so on until e :565::: = 3 3. De nition of the Log (a) If m = e p, then p = ln m: The log of m is p: (b) Or, ln m = p which is equivalent to e p = m (c) Which looks like:
A.5. ** NATURAL LOGARITHMS v 4. * Log of Products (a) Let m = e p () ln m = p n = e q () ln n = q (b) Then, multiply m times n: m n = e p e q = e (p+q) (c) Taking the log of both sides: ln (m n) = ln e (p+q) = p + q = ln m + ln n (d) For example: 3 = 6 ln ( 3) = ln + ln 3 = = 0: 6935::: + :0986::: = :798::: = ln 6 5. * Log of Quotients m ln n ln 3 The logit: = ln m ln n = : 40547 = ln 3 ln = : 0986 : 6935 p ln p = 6. Inverse operations (a) ln(k) is that power of e that equals k: k = e ln k (b) ln(e k ) is that power of e that equals e k, namely k: ln e k = k
vi APPENDIX A. MATH REVIEWS 03JAN007 (c) and e ln k = k 7. Log of Power 8. Example from Regression ln m n = n ln m ln 3 = ln 9 = : 97 = ln 3 = (: 0986) (a) Assume that (b) Taking logs: y = x x " ln y = ln x x " = ln + ln x + ln x + ln " = ln + ln x + ln x + " = + x + x + " A.6 Vector Algebra. Consider the regression equation for observation i: y i = 0 + x i + x i + " i Vector multiplication allows us to write this more simply.. For example, let 0 = 0 and x = x x, then 0 x = x x @ 0 A = 0 + x + x 3. More generally, consider K and x K, then by de nition: x = 0 + KX i x i = 0 + x + x + 3 x 3 + i=
A.7. PROBABILITY DISTRIBUTIONS vii A.7 Probability Distributions I Let X be a random variable with discrete outcomes x. The frequency of those outcomes is the probability distribution: f(x) = Pr(X = x) Bernoulli Distribution For example, let y indicate the outcome of a fair coin. Then, y = 0 or, and Pr (y = 0) = :4 and Pr (y = ) = :6 I For all probability distributions:. All probabilities are between zero and one: 0 f(x). Probabilities sum to one: P x f(x i) =
viii APPENDIX A. MATH REVIEWS 03JAN007 I For a continuous random variable, f(x) is called a probability density function or pdf.. f(x) = 0. Why? Pick any two numbers. Can you nd a number in between them?. Pr(a x b) = 3. Z f(t)dt = Normal Distribution Z b a f(x)dx 0. The pdf mean and standard deviation, x N(; ) ; is:! f(x j ; ) = p (x ) exp (A.). This de nes the classic bell curve: 3. If = 0 and = : (x) = f(x j 0; ) = p x exp (A.)
A.7. PROBABILITY DISTRIBUTIONS ix A.7. Cumulative Distribution Function (cdf) I The cdf is the probability of a value up to or equal to a speci c value. For discrete random variables: F (x) = P Xx f(x) = Pr (X x) For a continuous variable: F (x) = I For the cdf:. 0 F (x).. If x > y, then F (x) F (y). 3. F ( ) = 0 and F () =. Z x f(t)dt = Pr (X x) A.7. * Computing the Area Within a Distribution I Consider the distribution f (x), where F (x) = Pr (X x): Pr (a X b) = Pr (X b) Pr (X a) = F (b) F (a)
x APPENDIX A. MATH REVIEWS 03JAN007 A.7.3 * Expectation I The mean of N sample values of X is: For example: + + 4 + 0 4 = x = P N i= X i N + 4 + 0 = 4 4 4 4 I The expectation is de ned in terms of the population: For discrete variables: E(X) = X x f(x)x = X x Pr(X = x)x For continuous variables: Z E(X) = f(x)x dx x * Example of Expectation of Binary Variable If X has values 0 and with probabilities 4 and 3 4, then E(X) = 0 + 3 = 3 4 4 4 = Pr (x = ) = [Value Pr (Value )] + [Value Pr (Value )]
A.7. PROBABILITY DISTRIBUTIONS xi * Expectation of Sums I If X and Y are random variables, and a, b and c are constants, then E(a + bx + cy ) = a + be(x) + ce(y ) (A.3) I Example: Let Then KX y i = + k x ki + " i k=! KX E(y i ) = E + k x ki + " i = E () + E k=! KX k x ki + E (" i ) k= KX = + k E(x ik ) k= Conditional Expectations I Conditioning means holding some things constant while something else changes. I Example: Let $ be income. E($) tells us the mean $; but is not useful for telling us how other variables a ect $. Let S be the sex of the respondent. We might compute: E($ j S = female) = Expected $ for females This allows us to see how the expectation varies by the level of other variables. I Example: If y = x + ", then E(y j x) = E(x + ") = E(x) + E(") = x
xii APPENDIX A. MATH REVIEWS 03JAN007 A.7.4 The Variance I The variance is de ned as P N s i= = (x i x) N I Variance for a Population: Let f(x) = Pr(X = x). If x is discrete: Var(X) = X x [x E(x)] f(x) (A.4) If x is continuous: Z Var(X) = [x E(x)] f(x)dx (A.5) x Example of Variance of Binary Variable If X has values 0 and with probabilities 4 and 3 4, then E(X) = 3 4, and! 3 Var(X) = 0 + 4 4 9 = 6 + 4 6 3 4 = E (X) [ E (X)] * Variance of a Linear Transformation! 3 3 4 4 I Let X be a random variable, and a and b be constants. Then, = 9 64 + 3 64 = 64 = 3 6 Var(a + bx) = b Var(X) (A.6) * Variance of a Sum I Let X and Y be two random variables with constants a and b: Var(aX + by ) = a Var(X) + b Var(Y ) + abcov(x; Y ) (A.7) I Let Y = P K i= X i. If the X s are uncorrelated, then KX Var(Y ) = Var( X i ) = i= KX Var(X i ) i= (A.8)
A.8. **RESCALING VARIABLES xiii A.8 **Rescaling Variables Often we want to use addition and multiplication to change a variable with mean and variance into a variable with mean 0 and variance. This is called rescaling.. Consider X where E (x) = and Var (x) =. By subtracting the mean, the expectation becomes zero: E (x ) = E (x) E () = = 0 3. But the variance is unchanged: Var (x ) = Var (x) = 4. Dividing by : x E = E x = E (x) = = 5. Subtracting and dividing by does not change the mean: x E = E (x ) = 0 = 0 (A.9) 6. But, the variance becomes one: x Var = Var (x ) = Var (x) = (A.0)
xiv APPENDIX A. MATH REVIEWS 03JAN007 Stata: Standardizing Variables. use science, clear. sum pub9 Variable Obs Mean Std. Dev. Min Max ---------+----------------------------------------------------- pub9 308 4.5987 5.3534 0 33. gen p9_mn = pub9 - r(mean). sum p9_mn Variable Obs Mean Std. Dev. Min Max ---------+----------------------------------------------------- p9_mn 308-5.7e-09 5.3534-4.5987 8.4870. gen p9_sd = p9_mn/5.3534. gen p9_sd = (pub9-4.5987)/5.3534. egen p9_sdez = std(pub9). sum p9_sd p9_sd p9_sdez Variable Obs Mean Std. Dev. Min Max ---------+----------------------------------------------------- p9_sd 308 4.0e-09.9999999 -.849085 5.359604 p9_sd 308.8e-0.9999999 -.849085 5.359604 p9_sdez 308-6.77e-0 -.849085 5.359604
A.9. DISTRIBUTIONS xv A.9 Distributions A.9. Bernoulli I X has a Bernoulli distribution if it has two possible outcomes: Pr(X = ) = p and Pr(X = 0) = p I Then: I That is: f(x j p) = p x ( p) x = Pr(X = x j p) I It can be shown that f(0 j p) = p 0 ( p) = p and f( j p) = p ( p) 0 = p I Note how the variance is related to the mean: E(X) = p and Var (X) = p ( p) (A.) E(X) = p Var (X) = p ( p) 0. 0.090 0. 0.60 0.3 0.0 0.4 0.40 0.5 0.50 0.6 0.40 0.7 0.0 0.8 0.60 0.9 0.090
xvi APPENDIX A. MATH REVIEWS 03JAN007 A.9. Normal I The pdf for a normal distribution with mean and standard deviation is! f(x j ; ) = p (x ) exp (A.) I If x is distributed normally with mean and standard deviation : x N ; I The cdf is de ned as F (x j ; ) = Z x f(t j ; )dt
A.9. DISTRIBUTIONS xvii Standardized Normal I If x N(0; ), we de ne: x pdf: (x) = p exp cdf: (x) = R x (t)dt I You can move from an unstandardized to a standardized normal distribution. Let x N(0; ) Then, f(x j 0; ) = = x p exp = x p exp x = (A.3) Area Under the Curve I If x N(0; ), then Pr (a x b) = (b) (a) (A.4) Linear Transformation of a Normal I If I Then Sums of Normals x N ; a + bx N a + b; b (A.5) I Let Cor(x ; x ) =, where x N ; and x N ; I Then x + x N([ + ] ; + + ) I When = 0: x + x N([ + ] ; + ) (A.6)
xviii A.9.3 I Let i Chi-square (i = to df) be independent, standard normal variates. APPENDIX A. MATH REVIEWS 03JAN007 I De ne: Xdf dfx i= i df I The chi-square distribution is de ned as the sum of independent squared normal variables. I The mean and variance: E Xdf = df and Var X df = df Adding Chi-squares Let x df x and y df y If x and y are independent: x + y df x +df y Shape With df, the distribution is highly skewed. As df!, the chi-square becomes distributed normally. Question from Intro to Statistics X = X all cells (obs exp) exp Consider the chi-square test in contingency tables: X df with df = (#rows )(#columns ) Why would this be distributed as chi-square? Why those degrees of freedom? A.9.4 F-distribution I Let X and X be independent chi-square variables with degrees of freedom r and r. I The F-distribution is de ned as: F r ;r X =r X =r
A.9. DISTRIBUTIONS xix A.9.5 t-distribution I Consider z and x df, where z and x are independent. I Then the t-distribution with df degrees of freedom is de ned as: t df z p x=df A.9.6 Relationships among normal, t, chi-square and F. z = t. z = X = F ; = t 3. t df = F ;df 4. X df df = F r;
xx APPENDIX A. MATH REVIEWS 03JAN007 A.0 Calculus I The two central ideas in calculus are the derivative and the integral. I Derivative: The derivative is the slope of a curve y = f (x): dy dx = f 0 (x) (A.7) I The second derivative indicates how quickly the slope of the curve is changing: d y dx = d dy dx = f 00 (x) (A.8) dx I If the curve is de ned as y = f(x; z), we write the partial derivative with respect to x as @f(x; z) (A.9) @x Imagine half of a hard boiled egg setting on a table; slice it from the top to the table. The partial derivative is the slope on the resulting curve. I Integral: The integral is the area under a curve. I For example, if a curve is de ned as y = f(x), the area under the curve from point a to point b is computed with the integral: Z a b f (t) dt
A.. MATRIX ALGEBRA xxi A. Matrix Algebra A.. Matrix Basic De nitions is an array of numbers, arranged in rows and columns: a a A = a 3 a a a 3 A.. Transposing a Matrix I The transpose is indicated by the prime or superscript T. For example: A 0 or A T : If A =, then A 0 = I Transposing the Transpose I If A is a symmetric matrix, then A 0 = A A..3 I Addition: A 00 = A Addition and Subtraction 4 5 I Transposes of added matrices: + A + B = fa rc + b rc g 3 7 = + + 3 4 + 7 5 + (A.30) 4 5 + I Subtraction of matrices: (A + B) 0 = A 0 + B 0 (A.3) 0 0 0 3 = + 4 5 7 4 7 = + = 5 3 5 6 3 7 4 5 A B = fa rc b rc g = 3 7 3 4 7 5 A..4 Scalar Multiplication A = fa rc g = f a rc g 3 3 3 = 4 5 3 4 3 5
xxii A..5 Vector Matrix Multiplication is a matrix with one dimension equal to one. APPENDIX A. MATH REVIEWS 03JAN007 I A column vector is an R matrix: 3 c = 4 5 or c 0 = 3 3 I A row vector is a C matrix: r = 3 4 * Vector Multiplication Consider K and x K, then by de nition: x = 3X i x i = x + x + 3 x 3 i= I For example, let 0 = 0 and x = x x, then 0 0 x = x x @ A = 0 + x + x Matrix Multiplication For A RK and B KC, the matrix product C RC = AB equals: ( X K fc rc g = i= a ri b ic ) I Note that element c rc is the vector multiplication of row r from A and column c from B. I Example: 3 4 5 6 4 a b c d e f g h i I Example from Regression: 3 5 = a + d + 3g b + e + 3h c + f + 3i 4a + 5d + 6g 4b + 5e + 6h 4c + 5f + 6i 6 4 y. y N y = X + " 3 x x 7 6 5 = 4... x N x N = 6 4 3 7 5 4 0 3 5 + 0 + x + x + ". 0 + x N + x N + " N 6 4 3 7 5 ". " N 3 7 5
A.. MATRIX ALGEBRA xxiii A..6 Inverse I An identity matrix is a square matrix with s on the diagonal, and 0 s elsewhere. I If A is square, then A is the inverse of A if and only if AA = I (A.3) 4 5 3 3 3 3 = 0 0. If A exists, it is unique.. If A does not exist, A is called singular. A..7 Rank I Rank is the size of the largest submatrix that can be inverted. I A matrix is of full rank if the rank is equal to the minimum of the number of rows and columns. I Problems occur in estimation when a matrix is encountered that is not of full rank. I When this occurs, messages such as the following are generated: Matrix is not of full rank. Singular matrix encountered. Matrix cannot be inverted. An inverse does not exist.