Bayesian Analysis LEARNING OBJECTIVES. Calculating Revised Probabilities. Calculating Revised Probabilities. Calculating Revised Probabilities

Transcription

1 Valua%on and pricing (November 5, 2013) LEARNING OBJECTIVES Lecture 7 Decision making (part 3) Regression theory Olivier J. de Jong, LL.M., MM., MBA, CFD, CFFA, AA 1. List the steps of the decision-making process. 2. Describe the types of decision-making environments. 3. Make decisions under uncertainty. 4. Use probability values to make decisions under risk. 5. Develop accurate and useful decision trees. 6. Revise probabilities using Bayesian analysis. 7. Use computers to solve basic decision-making problems. 8. Understand the importance and use of utility theory in decision making. Copyright 2015 Pearson Education, Inc. 3 2 Bayesian Analysis Many ways of getting probability data Management s experience and intuition Historical data Computed from other data using Bayes theorem Bayes theorem incorporates initial estimates and information about the accuracy of the sources Allows the revision of initial estimates based on new information Calculating Revised Probabilities Four conditional probabilities for Thompson Lumber P(favorable market(fm) survey results positive) 0.78 P(unfavorable market(um) survey results positive) 0.22 P(favorable market(fm) survey results negative) 0.27 P(unfavorable market(um) survey results negative) 0.73 Prior probabilities P(FM) 0.50 P(UM) Calculating Revised Probabilities Calculating Revised Probabilities TABLE 3.16 Market Survey Reliability Calculating posterior probabilities RESULT OF SURVEY Positive (predicts favorable market for product) Negative (predicts unfavorable market for product) FAVORABLE MARKET (FM) P (survey positive FM) 0.70 P (survey negative FM) 0.30 STATE OF NATURE UNFAVORABLE MARKET (UM) P (survey positive UM) 0.20 P (survey negative UM) 0.80 P(B A) P(A) P(A B) P(B A) P(A)+ P(B A ") P( A ") where A, B any two events A complement of A A favorable market B positive survey

2 Calculating Revised Probabilities P(FM survey positive) P(survey positive FM)P(FM) P(survey positive FM)P(FM)+ P(survey positive UM)P(UM) (0.70)(0.50) (0.70)(0.50)+(0.20)(0.50) P(UM survey positive) P(survey positive UM)P(UM) P(survey positive UM)P(UM)+ P(survey positive FM)P(FM) (0.20)(0.50) (0.20)(0.50)+(0.70)(0.50) Calculating Revised Probabilities TABLE 3.17 Probability Revisions Given a Positive Survey STATE OF NATURE CONDITIONAL PROBABILITY P(SURVEY POSITIVE STATE OF NATURE) PRIOR PROBABILITY POSTERIOR PROBABILITY JOINT PROBABILITY P(STATE OF NATURE SURVEY POSITIVE) FM / UM / P(survey results positive) Calculating Revised Probabilities P(FM survey negative) P(survey negative FM)P(FM) P(survey negative FM)P(FM)+ P(survey negative UM)P(UM) (0.30)(0.50) (0.30)(0.50)+(0.80)(0.50) P(UM survey negative) P(survey negative UM)P(UM) P(survey negative UM)P(UM)+ P(survey negative FM)P(FM) (0.80)(0.50) (0.80)(0.50)+(0.30)(0.50) Calculating Revised Probabilities TABLE 3.18 Probability Revisions Given a Negative Survey STATE OF NATURE CONDITIONAL PROBABILITY P(SURVEY NEGATIVE STATE OF NATURE) PRIOR PROBABILITY POSTERIOR PROBABILITY JOINT PROBABILITY P(STATE OF NATURE SURVEY NEGATIVE) FM / UM / P(survey results positive) Potential Problems Using Survey Results We can not always get the necessary data for analysis Survey results may be based on cases where an action was taken Conditional probability information may not be as accurate as we would like Utility Theory Monetary value is not always a true indicator of the overall value of the result of a decision The overall value of a decision is called utility Economists assume that rational people make decisions to maximize their utility 3 11 Copyright 2015 Pearson Education, Inc. 3 12

3 Utility Theory Utility Theory FIGURE 3.6 Decision Tree for the Lottery Ticket Accept Offer Reject Offer $2,000,000 Heads (0.5) $0 Utility assessment assigns the worst outcome a utility of 0 and the best outcome a utility of 1 A standard gamble is used to determine utility values When you are indifferent, your utility values are equal Tails (0.5) EMV $2,500,000 $5,000,000 Copyright 2015 Pearson Education, Inc Copyright 2015 Pearson Education, Inc Utility Theory Utility Theory FIGURE 3.7 Standard Gamble for Utility Assessment Alternative 1 Alternative 2 (p) (1 p) Best Outcome Utility 1 Worst Outcome Utility 0 Other Outcome Utility? Expected utility of alternative 2 Expected utility of alternative 1 Utility of other outcome (p)(utility of best outcome, which is 1) + (1 p)(utility of the worst outcome, which is 0) Utility of other outcome (p)(1) + (1 p)(0) p Copyright 2015 Pearson Education, Inc Copyright 2015 Pearson Education, Inc Investment Example Construct a utility curve revealing preference for money between $0 and $10,000 A utility curve plots the utility value versus the monetary value An investment in a bank will result in $5,000 An investment in real estate will result in $0 or $10,000 Unless there is an 80% chance of getting $10,000 from the real estate deal, prefer to have her money in the bank If p 0.80, Jane is indifferent between the bank or the real estate investment Copyright 2015 Pearson Education, Inc Investment Example Figure 3.8 Utility of $5,000 Invest in Real Estate Invest in Bank p 0.80 (1 p) 0.20 $10,000 U($10,000) 1.0 $0 U($0.00) 0.0 $5,000 U($5,000) p 0.80 Utility for $5,000 U($5,000) pu($10,000) + (1 p)u($0) (0.8)(1) + (0.2)(0) 0.8 Copyright 2015 Pearson Education, Inc. 3 18

4 Investment Example Assess other utility values Utility for $7, Utility for $3, Use the three different dollar amounts and assess utilities FIGURE 3.9 Utility Curve U ($10,000) Utility U ($7,000) U ($5,000) U ($3,000) Utility Curve U ($0) 0 Copyright 2015 Pearson Education, Inc $0 $1,000 $3,000 $5,000 $7,000 $10,000 Monetary Value 3 20 Utility Curve Preferences for Risk Typical of a risk avoider Less utility from greater risk Avoids situations where high losses might occur As monetary value increases, utility curve increases at a slower rate A risk seeker gets more utility from greater risk As monetary value increases, the utility curve increases at a faster rate Risk indifferent gives a linear utility curve FIGURE 3.10 Utility Risk Avoider Risk Indifference Risk Seeker Monetary Outcome Utility as a Decision-Making Criteria Utility as a Decision-Making Criteria Once a utility curve has been developed it can be used in making decisions Replaces monetary outcomes with utility values Expected utility is computed instead of the EMV Mark Simkin loves to gamble A game tossing thumbtacks in the air If the thumbtack lands point up, Mark wins $10,000 If the thumbtack lands point down, Mark loses $10,000 Mark believes that there is a 45% chance the thumbtack will land point up Should Mark play the game (alternative 1)?

5 Utility as a Decision-Making Criteria FIGURE 3.11 Decision Facing Mark Simkin Alternative 1 Mark Plays the Game Alternative 2 Tack Lands Point Up (0.45) Tack Lands Point Down (0.55) $10,000 $10,000 Utility as a Decision-Making Criteria Step 1 Define Mark s utilities U( $10,000) 0.05 U($0) 0.15 U($10,000) 0.30 Utility Mark Does Not Play the Game $0 FIGURE $20,000 $10,000 $0 $10,000 $20,000 Monetary Outcome Utility as a Decision-Making Criteria Utility as a Decision-Making Criteria Step 2 Replace monetary values with utility values E(alternative 1: play the game) (0.45)(0.30) + (0.55)(0.05) E(alternative 2: don t play the game) 0.15 FIGURE 3.13 Using Expected Utilities E Alternative 1 Mark Plays the Game Tack Lands Point Up (0.45) Tack Lands Point Down (0.55) Utility Alternative 2 Don t Play Introduction Regression Models Regression analysis very valuable tool for a manager Understand the relationship between variables Predict the value of one variable based on another variable Simple linear regression models have only two variables Multiple regression models have more than one independent variable Copyright 2015 Pearson Education, Inc. Copyright 2015 Pearson Education, Inc. 4 30

6 Introduction Variable to be predicted is called the dependent variable or response variable Value depends on the value of the independent variable(s) Explanatory or predictor variable Scatter Diagram Scatter diagram or scatter plot often used to investigate the relationship between variables Independent variable normally plotted on axis Dependent variable normally plotted on Y axis Dependent variable Independent + variable Independent variable Copyright 2015 Pearson Education, Inc Copyright 2015 Pearson Education, Inc renovates old homes The dollar volume of renovation work is dependent on the area payroll TABLE 4.1 TRIPLE A S SALES LOCAL PAYROLL ($100,000s) ($100,000,000s) Copyright 2015 Pearson Education, Inc Figure 4.1 Scatter Diagram Sales ($100,000) Payroll ($100 million) Copyright 2015 Pearson Education, Inc Simple Linear Regression Regression models used to test relationships between variables Random error Y β 0 + β 1 +ε where Y dependent variable (response) independent variable (predictor or explanatory) β 0 intercept (value of Y when 0) β 1 slope of the regression line ε random error Simple Linear Regression True values for the slope and intercept are not known Estimated using sample data ˆ Y b 0 + b 1 where ^ Y predicted value of Y b 0 estimate of β 0, based on sample results b 1 estimate of β 1, based on sample results Copyright 2015 Pearson Education, Inc Copyright 2015 Pearson Education, Inc. 4 36

7 Predict sales based on area payroll Y Sales Area payroll The line Figure 4.1 minimizes the errors Error (Actual value) (Predicted value) e Y ˆ Y Regression analysis minimizes the sum of squared errors Least-squares regression Copyright 2015 Pearson Education, Inc Formulas for simple linear regression, intercept and slope ˆ Y b 0 + b 1 average (mean) of values n Y Y average (mean) of Y values n ( )(Y Y ) b 1 ( ) 2 b 0 Y b 1 Copyright 2015 Pearson Education, Inc ΣY 42 TABLE 4.2 Regression calculations Y ( ) 2 ( )(Y Y) 6 3 (3 4) 2 1 (3 4)(6 7) (4 4) 2 0 (4 4)(8 7) (6 4) 2 4 (6 4)(9 7) (4 4) 2 0 (4 4)(5 7) (2 4) 2 4 (2 4)(4.5 7) (5 4) 2 1 (5 4)(9.5 7) 2.5 Y 42/6 7 Σ 24 24/6 4 Σ( ) 2 10 Σ( )(Y Y) 12.5 Regression calculations Y Y ( )(Y Y ) b ( ) b 0 Y b 1 7 (1.25)(4) 2 Therefore ˆ Y Copyright 2015 Pearson Education, Inc Copyright 2015 Pearson Education, Inc Regression calculations sales 4 2 Y (payroll) Y If the payroll next ( )(Y year Y ) is $600 b ( ) million Y ˆ (6) 9.5 or $ 950,000 b 0 Y b 1 7 (1.25)(4) 2 Therefore ˆ Y Measuring the Fit of the Regression Model Regression models can be developed for any variables and Y How helpful is the model in predicting Y? With average error positive and negative errors cancel each other out Three measures of variability SST Total variability about the mean SSE Variability about the regression line SSR Total variability that is explained by the model Copyright 2015 Pearson Education, Inc Copyright 2015 Pearson Education, Inc. 4 42

8 Measuring the Fit of the Regression Model Sum of squares total SST (Y Y ) 2 Sum of squares error SSE e 2 (Y Y ˆ ) 2 Sum of squares regression SSR ( Y ˆ Y ) 2 An important relationship SST SSR + SSE Measuring the Fit of the Regression Model TABLE 4.3 Sum of Squares for Y (Y Y) 2 ^ ^ Y (Y Y) 2 ^ (Y Y) (6 7) (3) (8 7) (4) (9 7) (6) (5 7) (4) (4.5 7) (2) (9.5 7) (5) (Y Y) ^ (Y Y) ^ (Y Y) Y 7 SST 22.5 SSE SSR Copyright 2015 Pearson Education, Inc Copyright 2015 Pearson Education, Inc Measuring the Fit of the Regression Model Sum of squares total SST For (YTriple Y ) 2 A Construction Sum of squares error SST 22.5 SSE SSE e 2 (Y ˆ Y ) SSR Sum of squares regression SSR ( Y ˆ Y ) 2 An important relationship SST SSR + SSE Measuring the Fit of the Regression Model FIGURE 4.2 Deviations from the Regression Line and from the Mean Sales ($100,000) Y ˆ Y ˆ Y Y Y Y ˆ Y Payroll ($100 million) Y Copyright 2015 Pearson Education, Inc Copyright 2015 Pearson Education, Inc Coefficient of Determination The proportion of the variability in Y explained by the regression equation The coefficient of determination is r 2. r 2 SSR SSE 1 SST SST For r Coefficient of Determination The proportion of the variability in Y explained by the regression equation The coefficient of determination is r 2. r 2 SSR About SSE69% of the 1 SST variability SST in Y is explained by the equation based on For payroll () r Copyright 2015 Pearson Education, Inc Copyright 2015 Pearson Education, Inc. 4 48

9 Correlation Coefficient An expression of the strength of the linear relationship Always between +1 and 1 The correlation coefficient is r FIGURE 4.3 Four Values of the Correlation Coefficient Y Y r ± r 2 For Y (a) Perfect Positive Correlation: r +1 Y (b) Positive Correlation: 0 < r < 1 r Copyright 2015 Pearson Education, Inc (c) No Correlation: r 0 (d) Perfect Negative Correlation: r 1 Copyright 2015 Pearson Education, Inc Assumptions of the Regression Model With certain assumptions about the errors, statistical tests can be performed to determine the model s usefulness 1. Errors are independent 2. Errors are normally distributed 3. Errors have a mean of zero 4. Errors have a constant variance A plot of the residuals (errors) often highlights glaring violations of assumptions Residual Plots FIGURE 4.4A Pattern of Errors Indicating Randomness Error Copyright 2015 Pearson Education, Inc Copyright 2015 Pearson Education, Inc Residual Plots Residual Plots FIGURE 4.4B Nonconstant error variance FIGURE 4.4C Errors Indicate Relationship is not Linear Error Error Copyright 2015 Pearson Education, Inc Copyright 2015 Pearson Education, Inc. 4 54

10 Estimating the Variance Errors are assumed to have a constant variance (σ 2 ), usually unknown Estimated using the mean squared error (MSE), s 2 s 2 MSE SSE n k 1 where n number of observations in the sample k number of independent variables Estimating the Variance For s 2 MSE SSE n k Estimate the standard deviation, s The standard error of the estimate or the standard deviation of the regression s MSE Copyright 2015 Pearson Education, Inc Copyright 2015 Pearson Education, Inc Testing the Model for Significance When the sample size is too small, you can get good values for MSE and r 2 even if there is no relationship between the variables Testing the model for significance helps determine if the values are meaningful Performing a statistical hypothesis test Testing the Model for Significance We start with the general linear model Y β 0 + β 1 +ε If β 1 0, the null hypothesis is that there is no relationship between and Y The alternate hypothesis is that there is a linear relationship (β 1 0) If the null hypothesis can be rejected, we have proven there is a relationship We use the F statistic Copyright 2015 Pearson Education, Inc Copyright 2015 Pearson Education, Inc Testing the Model for Significance The F statistic is based on the MSE and MSR MSR SSR k where k number of independent variables in the model The F statistic is F MSR MSE Describes an F distribution with: degrees of freedom for the numerator df 1 k degrees of freedom for the denominator df 2 n k 1 Copyright 2015 Pearson Education, Inc Testing the Model for Significance If there is very little error, MSE would be small and the F statistic would be large model is useful If the F statistic is large, the significance level (p-value) will be low, unlikely would have occurred by chance When the F value is large, we can reject the null hypothesis and accept that there is a linear relationship between and Y and the values of the MSE and r 2 are meaningful Copyright 2015 Pearson Education, Inc. 4 60

11 Steps in a Hypothesis Test 1. Specify null and alternative hypotheses H 0 0 H Select the level of significance (α) Common values are 0.01 and Calculate the value of the test statistic F MSR MSE Steps in a Hypothesis Test 4. Make a decision using one of the following methods a) Reject the null hypothesis if the test statistic is greater than the F value from the table in Appendix D. Otherwise, do not reject the null hypothesis: Reject if F calculated > F α,df1,df 2 df 1 k df 2 n k 1 b) Reject the null hypothesis if the observed significance level, or p-value, is less than the level of significance (α). Otherwise, do not reject the null hypothesis: p-value P(F > calculated test statistic) Reject if p-value < α Copyright 2015 Pearson Education, Inc Copyright 2015 Pearson Education, Inc Step 1 H 0 0 (no linear relationship between and Y) H 1 0 (linear relationship exists between and Y) Step 2 MSR SSR k Select α 0.05 Step 3 Calculate the value of the test statistic F MSR MSE Copyright 2015 Pearson Education, Inc Step 4 Reject the null hypothesis if the test statistic is greater than the F value in Appendix D df 1 k 1 df 2 n k The value of F associated with a 5% level of significance and with degrees of freedom 1 and 4 is found in Appendix D. F 0.05,1, F calculated 9.09 Reject H 0 because 9.09 > 7.71 Copyright 2015 Pearson Education, Inc FIGURE 4.5 We can conclude there is a statistically significant relationship between and Y The r 2 value of 0.69 means about 69% of the variability in sales (Y) is explained by local payroll () Analysis of Variance (ANOVA) Table With software models, an ANOVA table is typically created that shows the observed significance level (p-value) for the calculated F value This can be compared to the level of significance (α) to make a decision TABLE 4.4 DF SS MS F SIGNIFICANCE Regression k SSR MSR SSR/k MSR/MSE P(F > MSR/MSE) 0.05 F Residual n - k - 1 SSE MSE SSE/(n - k - 1) Total n - 1 SST Copyright 2015 Pearson Education, Inc

12 Multiple Regression Analysis Extensions to the simple linear model Models with more than one independent variable Y β 0 + β β β k k + ε where Y dependent variable (response variable) i i th independent variable (predictor or explanatory variable) β 0 intercept (value of Y when all i 0) β i coefficient of the i th independent variable k number of independent variables ε random error Multiple Regression Analysis To estimate these values, a sample is taken the following equation developed ˆ Y b 0 + b b b k k Where Yˆ predicted value of Y b 0 sample intercept (an estimate of β 0 ) b i sample coefficient of the ith variable (an estimate of β i ) Jenny Wilson Realty Develop a model to determine the suggested listing price for houses based on the size and age of the house Y ˆ b 0 + b b 2 2 where Yˆ predicted value of dependent variable (selling price) b 0 Y intercept 1 and 2 value of the two independent variables (square footage and age) respectively b 1 and b 2 slopes for 1 and 2 respectively Selects a sample of houses that have sold recently and records the data Jenny Wilson Real Estate Data TABLE 4.5 SELLING PRICE ($) SQUARE FOOTAGE AGE 95,000 1, Good CONDITION 119,000 2, Excellent 124,800 1, Excellent 135,000 1, Good 142,000 1, Mint 145,000 1, Mint 159,000 1, Mint 165,000 2, Excellent 182,000 2, Mint 183,000 3, Good 200,000 2, Good 211,000 2, Good 215,000 3, Excellent 219,000 1, Mint Evaluating Multiple Regression Models Evaluating Multiple Regression Models Similar to simple linear regression models The p-value for the F test and r 2 interpreted the same The hypothesis is different because there is more than one independent variable The F test is investigating whether all the coefficients are equal to 0 at the same time To determine which independent variables are significant, tests are performed for each variable H 0 0 H 1 0 The test statistic is calculated and if the p- value is lower than the level of significance (α), the null hypothesis is rejected

13 Jenny Wilson Realty Full model is statistically significant Useful in predicting selling price p-value for F test r Jenny Wilson Realty Both square footage and age are helpful in predicting the Full model is statistically significant selling price Useful in predicting selling price p-value for F test r Are both variables significant? For 1 (square footage) H 0 0 H 1 0 Are both variables significant? For 1 square footage H 0 0 H 1 0 For α 0.05, p-value null hypothesis is rejected For α 0.05, p-value null hypothesis is rejected For 1 (age) For 1 age For α 0.05, p-value null hypothesis is rejected For α 0.05, p-value null hypothesis is rejected Binary or Dummy Variables Binary (or dummy or indicator) variables are special variables created for qualitative data A dummy variable is assigned a value of 1 if a particular condition is met and a value of 0 otherwise The number of dummy variables must equal one less than the number of categories of the qualitative variable Jenny Wilson Realty A better model can be developed if information about the condition of the property is included 3 1 if house is in excellent condition 0 otherwise 4 1 if house is in mint condition 0 otherwise Two dummy variables are used to describe the three categories of condition No variable is needed for good condition since if both 3 and 4 0, the house must be in good condition Model Building The best model is a statistically significant model with a high r 2 and few variables As more variables are added to the model, the r 2 value increases For this reason, the adjusted r 2 value is often used to determine the usefulness of an additional variable The adjusted r 2 takes into account the number of independent variables in the model The formula for r 2 Model Building r 2 SSR SSE 1 SST SST The formula for adjusted r 2 Adjusted r 2 SSE / (n k 1) 1 SST / (n 1) As the number of variables increases, the adjusted r 2 gets smaller unless the increase due to the new variable is large enough to offset the change in k

14 Model Building The formula for r 2 In general, if a new variable r 2 SSR increases the adjusted r SSE 2, it should probably 1 be included in the model SST SST The formula for adjusted r 2 Adjusted r 2 SSE / (n k 1) 1 SST / (n 1) As the number of variables increases, the adjusted r 2 gets smaller unless the increase due to the new variable is large enough to offset the change in k Model Building Stepwise regression systematically adds or deletes independent variables A forward stepwise procedure puts the most significant variable in first, adds the next variable that will improve the model the most Backward stepwise regression begins with all the independent variables and deletes the least helpful Model Building Nonlinear Regression In some cases variables contain duplicate information When two independent variables are correlated, they are said to be collinear When more than two independent variables are correlated, multicollinearity exists When multicollinearity is present, hypothesis tests for the individual coefficients are not valid but the model may still be useful In some situations, variables are not linear Transformations may be used to turn a nonlinear model into a linear model * * * * * * * * * * * * * * * * * * * Linear relationship Nonlinear relationship Colonel Motors Use regression analysis to improve fuel efficiency Study the impact of weight on miles per gallon (MPG) TABLE 4.6 MPG WEIGHT (1,000 LBS.) MPG WEIGHT (1,000 LBS.) Colonel Motors FIGURE 4.6A Linear Model for MPG Data MPG Weight (1,000 lb.)

15 Colonel Motors FIGURE 4.6B Nonlinear Model for MPG Data MPG Weight (1,000 lb.) Colonel Motors The nonlinear model is a quadratic model The easiest approach develop a new variable New model 2 (weight) 2 ˆ Y b 0 + b b Cautions and Pitfalls Cautions and Pitfalls If the assumptions are not met, the statistical test may not be valid Correlation does not necessarily mean causation Multicollinearity makes interpreting coefficients problematic, but the model may still be good Using a regression model beyond the range of is questionable, as the relationship may not hold outside the sample data A t-test for the intercept (b 0 ) may be ignored as this point is often outside the range of the model A linear relationship may not be the best relationship, even if the F test returns an acceptable value A nonlinear relationship can exist even if a linear relationship does not Even though a relationship is statistically significant it may not have any practical value Remember Chapter 3: Where Prices Come From: The Interaction of Demand and Supply Read Quantitative Methods-module guide. Any questions please olivier.edu@gmail.com and make notes as you do so, in whatever way works best for you in terms of remembering information (your performance on this course is only assessed by exam). Copyright 2010 Pearson Education, Inc. Economics R. Glenn Hubbard, Anthony Patrick O Brien, 3e. 89 of 46