so that you can see that it explains some, but certainly not nearly all!, of the variation.

Transcription

1 Lecture Notes 9 Econ 20150, Prncples of Statstcs Kevn R Foster, CCNY Fall 2012 Multple Regresson more than one X varable Regressng just one varable on another can be helpful and useful (and provdes a great graphcal ntuton) but t doesn't get us very far. Here's a graph of the relaton of age and wage, so that you can see that t explans some, but certanly not nearly all!, of the varaton. Consder ths example, usng data from the March 2010 CPS. We lmt ourselves to only examnng people wth a non-zero annual wage/salary who are workng fulltme (WSAL_VAL > 0 & HRCHECK = 2). We look at the dfferent wages reported by people who label themselves as whte, Afrcan-Amercan, Asan, Natve Amercan, and Hspanc. There are

2 62,043 whtes, 9,101 Afrcan-Amercans, 4476 Asans, 2149 Natve Amercans, and 12,401 Hspancs n the data who fulfll ths condton. The average yearly salary for whtes s $50,782; for Afrcan-Amercans t s $39,131; for Asans $57,541; for Natve Amercans $38,036; for Hspancs t s $36,678. Conventonal statstcal tests fnd that these averages are sgnfcantly dfferent. Does ths prove dscrmnaton? No; there are many other reasons why groups of people could have dfferent ncomes such as educatonal level or age or a multtude of other factors. (But t s not nconsstent wth a hypothess of racsm: remember the dfference, when evaluatng hypotheses, between 'not rejectng' or 'acceptng'). We mght reasonably break these numbers down further. These groups of people are dfferent n a varety of ways. Ther average ages are dfferent between Hspancs, averagng years, and non-hspancs, averagng years. So how much of the wage dfference, for Hspancs, s due to the fact that they're younger? We could do an ANOVA on ths but that would omt other factors. The populatons also dfferent n gender ratos. For whtes, 57% were male; for Afrcan- Amercans 46% were male; for Hspancs 59% were male. Snce gender also affects ncome, we mght thnk some of the wage gap could be due, not to racal dscrmnaton, but to gender dscrmnaton. But then they're also dfferent n educatonal attanment! Among the Hspanc workers, 30% had not fnshed hgh school; for Afrcan-Amercans 8.8% had not; for whtes 9% had not fnshed wth a dploma. And 12% of whtes had an advanced degree whle 8.3% of Afrcan Amercans and 4.2% of Hspancs had such credentals. The dfferent fractons n educatonal attanment add credblty to the hypothess that not all racal/ethnc varaton means dscrmnaton (n the labor market, at least there could be dscrmnaton n educaton so certan groups get less or worse educaton). Fnally they're dfferent n what secton of the country they lve n, as measured by Census regon. So how can we keep all of these dfferent factors straght? Multple Regresson From the standpont of just usng SPSS, there s no dfference for the user between a unvarate and multvarate lnear regresson. Agan use "Analyze\ Regresson\ Lnear..." but then add a bunch of varables to the "Independent(s)" box. In formulas, model has k explanatory varables for each of 1,2, n have n > k) y x x x 0 1 1, 2 2, k k, observatons (must

3 Each coeffcent estmate, notated as ˆ j, has standardzed dstrbuton as t wth (n k) degrees of freedom. Each coeffcent represents the amount by whch the y would be expected to change, for a y small change n the partcular x-varable (.e. j ). x Note that you must be a bt careful specfyng the varables. The CPS codes educatonal attanment wth a bunch of numbers from 31 to 46 but these numbers have no nherent meanng. So too race, geography, ndustry, and occupaton. If a person graduates hgh school then ther grade codng changes from 38 to 39 but ths must be coded wth a dummy varable. If a person moves from New York to North Dakota then ths ncreases ther state code from 36 to 38; ths s not the same change as would occur for someone movng from North Dakota to Oklahoma (40) nor s t half of the change as would occur for someone movng from New York to North Carolna (37). Each state needs a dummy varable. A multvarate regresson can control for all of the dfferent changes to focus on each tem ndvdually. So we mght model a person's wage/salary value as a functon of ther age, ther gender, race/ethncty (Afrcan-Amercan, Asan, Natve Amercan, Hspanc), f they're an mmgrant, sx educatonal varables (hgh school dploma, some college but no degree, Assocate's n vocatonal feld, Assocate's n academc feld, a 4-year degree, or advanced degree), f they're marred or dvorced/wdowed/separated, f they're a unon member, and f they're a veteran. Results (from the sample above, of March 2010 fulltme workers wth nonzero wage), are gven by SPSS as: j Model Summary Model R R Square Adjusted R Square Std. Error of the Estmate a a. Predctors: (Constant), Veteran (any), Afrcan Amercan, Educaton: Assocate n vocatonal, Unon member, Educaton: Assocate n academc, Natve Amercan Indan or Alaskan or Hawaan, Dvorced or Wdowed or Separated, Asan, Educaton: Advanced Degree, Hspanc, Female, Educaton: Some College but no degree, Demographcs, Age, Educaton: 4-yr degree, Immgrant, Marred, Educaton: Hgh School Dploma ANOVA b Model Sum of Squares df Mean Square F Sg.

4 1 Regresson 4.416E E a Resdual 1.704E E9 Total 2.146E a. Predctors: (Constant), Veteran (any), Afrcan Amercan, Educaton: Assocate n vocatonal, Unon member, Educaton: Assocate n academc, Natve Amercan Indan or Alaskan or Hawaan, Dvorced or Wdowed or Separated, Asan, Educaton: Advanced Degree, Hspanc, Female, Educaton: Some College but no degree, Demographcs, Age, Educaton: 4-yr degree, Immgrant, Marred, Educaton: Hgh School Dploma b. Dependent Varable: Total wage and salary earnngs amount - Person Coeffcents a Unstandardzed Coeffcents Standardzed Coeffcents Model B Std. Error Beta t Sg. 1 (Constant) Demographcs, Age Female Afrcan Amercan Asan Natve Amercan Indan or Alaskan or Hawaan Hspanc Immgrant Educaton: Hgh School Dploma Educaton: Some College but no degree Educaton: Assocate n vocatonal Educaton: Assocate n academc Educaton: 4-yr degree

5 Educaton: Advanced Degree Marred Dvorced or Wdowed or Separated Unon member Veteran (any) a. Dependent Varable: Total wage and salary earnngs amount - Person For the "Coeffcents" table, the "Unstandardzed coeffcent B" s the estmate of ˆ, the "Std. se ˆ. (In Error" of the unstandardzed coeffcent s the standard error of that estmate, economcs we don't generally use the standardzed beta, whch dvdes the coeffcent estmate by ˆ the standard error of X.) The "t" gven n the table s the t-statstc, t and "Sg." s ts p- se ˆ value the probablty, f the coeffcent were actually zero, of seeng an estmate as large as the one that you got. (Go back and revew f you don't remember all of the detals of ths.) So see Excel sheet to show how to get predcted wages for dfferent groups. Can then nterpret the resdual from the regresson. - Statstcal sgnfcance of coeffcent estmates s more complcated for multple regresson, we can ask whether a group of varables are jontly sgnfcant, whch takes a more complcated test. The dfference between the overall regresson ft and the sgnfcance of any partcular estmate s that a hypothess test of one partcular coeffcent tests f that parameter s zero; s ˆ β = 0? Ths uses the t-statstc t and compares t to a Normal or t dstrbuton se ˆ (dependng on the degrees of freedom). The test of the regresson sgnfcance tests f ALL of the slope coeffcents are smultaneously zero; f β 1 = β 2 = β 3 =... = β K = 0. The latter s much more restrctve. The predcted value of y s notated as ŷ, where yˆ ˆ ˆ ˆ ˆ 0 1x1 2x2 kxk. Its standard error s the standard error of the regresson, gven by SPSS as "Standard Error of the Estmate." y yˆ y ˆ ˆ ˆ ˆ The resdual s 0 1x1 2x2 kxk. The resdual of, for example, a wage regresson can be nterpreted as the part of the wage that s not explaned by the factors wthn the model.

6 Resduals are often used n analyses of productvty. Suppose I am analyzng a chan's stores to fgure out whch are managed best. I know that there are many reasons for varaton n revenues and cost so I can get data on those: how many workers are there and ther pay, the locaton of the store relatve to traffc, the rent pad, any sales or promotons gong on, etc. If I ŷ, of what proft would have run a regresson on all of those factors then I get an estmate, been expected, gven external factors. Then the dfference represents the unexplaned or resdual amount of varaton: some stores would have been expected to be proftable and are ndeed; some are not lvng up to potental; some would not have been expected to do so well but somethng s gong on so they're dong much better than expected. Why do we always leave out a dummy varable? Multcollnearty. OLS basc assumptons: o The condtonal dstrbuton of u gven X has a mean of zero. Ths s a complcated way of sayng somethng very basc: I have no addtonal nformaton outsde of the model, whch would allow me to make better guesses. It can also be expressed as mplyng a zero correlaton between X and u. We wll work up to other methods that ncorporate addtonal nformaton. o The X and errors are..d. Ths s often not precsely true; on the other hand t mght be roughly rght, and t gves us a place to start. o X and errors don't have values that are "too extreme." Ths s techncal (about exstence of fourth moments) and broadly true, whenever the X and Y data have a lmt on the amount of varaton, although there mght be partcular crcumstances where t s questonable (sometmes n fnance). So f these are true then the OLS are unbased and consstent. So E ˆ 0 0 and E ˆ 1 1. The normal dstrbuton, as the sample gets large, allows us to make hypothess tests about the values of the betas. In partcular, f you look back to the "eyeball" data at the begnnng, you wll recall that a zero value for the slope, 1, s mportant. It mples no relatonshp between the varables. So we wll commonly test the estmated values of aganst a null hypothess that they are zero. Nonlnear Regresson (more properly, How to Jam Nonlneartes nto a Lnear Regresson) X, X 2, X 3, X r ln(x), ln(y), both ln(y) & ln(x) dummy varables nteractons of dummes nteractons of dummy/contnuous nteractons of contnuous varables

7 There are many examples of, and reasons for, nonlnearty. In fact we can thnk that the most general case s nonlnearty and a lnear functonal form s just a convenent smplfcaton whch s sometmes useful. But sometmes the smplfcaton has a hgh prce. For example, my kds beleve that age and heght are closely related whch s true for ther sample (.e. mostly kds of a young age, for whom there s a tght relatonshp, plus 2 parents who are aged and tall). If my sample were all chldren then that mght be a decent smplfcaton; f my sample were adults then that's lousy. The usual justfcaton for a lnear regresson s that, for any dfferentable functon, the Taylor Theorem delvers a lnear functon as beng a close approxmaton but ths s only wthn a neghborhood. We need to work to get a good approxmaton. Nonlnear terms We can return to our regresson usng CPS data. Frst, we mght want to ask why our regresson s lnear. Ths s mostly convenence, and we can easly add non-lnear terms such as Age 2, f we thnk that the typcal age/wage profle looks lke ths: Wage Age So the regresson would be: Wage Age Age (where the term "..." ndcates "other stuff" that should be n the regresson). As we remember from calculus, dwage Age dage so that the extra boost n wage from another brthday mght fall as the person gets older, and even turn negatve f the estmate of 2 0 (a bt of algebra can solve for the top of the hll dwage by fndng the Age that sets 0 dage ). We can add hgher-order effects as well. Some labor econometrcans argue for ncludng Age 3 and Age 4 terms, whch can trace out some complcated wage/age profles. However we need to be careful of "overfttng" addng more explanatory varables wll never lower the R 2. Logarthms

8 Smlarly can specfy X or Y as ln(x) and/or ln(y). But we've got to be careful: remember from math (or theory of nsurance from Intermedate Mcro) that E[ln(Y)] IS NOT EQUAL TO ln(e[y])! In cases where we're regressng on wages, ths means that the log of the average wage s not equal to the average log wage. (Try t. Go ahead, I'll wat.) When both X and Y are measured n logs then the coeffcents have an easy economc dy 1 dx nterpretaton. Recall from calculus that wth y ln x and, so dy % x -- our dx x x usual frend, the percent change. So n a regresson where both X and Y are n logarthms, then y % y j s the elastcty of Y wth respect to X. x % x Also, f Y s n logs and D s a dummy varable, then the coeffcent on the dummy varable s just the percent change when D swtches from zero to one. So the choce of whether to specfy Y as levels or logs s equvalent to askng whether dummy varables are better specfed as havng a constant level effect (.e. women make $10,000 less than men) or havng a percent change effect (women make 25% less than men). As usual there s no general answer that one or the other s always rght! Recall our dscusson of dummy varables, that take values of just 0 or 1, whch we ll represent as D. Snce, unlke the contnuous varable Age, D takes just two values, t represents a shft of the constant term. So the regresson, Wage Age D u shows that people wth D=0 have ntercept of just 0, whle those wth D=1 have ntercept equal to Graphcally, ths s: We need not assume that the 3 term s postve f t were negatve, t would just shft the lne downward. We do however assume that the rate at whch age ncreases wages s the same for both genders the lnes are parallel.

9 Dummy Varables Interactng wth Other Explanatory Varables The assumpton about parallel lnes wth the same slopes can be modfed by addng nteracton terms: defne a varable as the product of the dummy tmes age, so the regresson s Wage 0 1Age 3D 4D Age u Wage Wage so that, for those wth D=0, as before = 1 but for those wth D=1, 14. Age Age Graphcally, so now the ntercepts and slopes are dfferent. So we mght wonder f men and women have a smlar wage-age profle. We could ft a number of possble specfcatons that are varatons of our basc model that wage depends on age and age-squared. The frst possble varaton s smply that: 2 Wage Age Age D u, whch allows the wage profle lnes to have dfferent ntercept-values but otherwse to be parallel (the same hump pont where wages have ther maxmum value), as shown by ths graph: w Age The next varaton would be to allow the lnes to have dfferent slopes as well as dfferent ntercepts:

10 Wage Age Age D 4D Age 5D Age u whch allows the two groups to have dfferent-shaped wage-age profles, as n ths graph: w Age (The wage-age profles mght ntersect or they mght not t depends on the sample data.) Ths specfcaton, wth a dummy varable multplyng each term: the constant and all the explanatory varables, s equvalent to runnng two separate regressons: one for men and one for women: D 0 D 1 Wage Age Age u male male male Wage Age Age e female female female female Where the new coeffcents are related to the old by the denttes: 0 0 3, female female 1 1 4, and Sometmes breakng up the regressons s easer, f there are large datasets and many nteractons. Testng f All the New Varable Coeffcents are Zero You're wonderng how to tell f all of these new nteractons are worthwhle. Smple: Hypothess Testng! There are varous formulas, some more complcated, but for the case of homoskedastcty the formula s relatvely smple. Why any formula at all why not look at the t-tests ndvdually? Because the ndvdual t-tests are askng f each ndvdual coeffcent s zero, not f t s zero and others as well are also zero. That would be a stronger test. To measure how much a group of varables contrbutes to the regresson, we look at the resdual values how much s stll unexplaned, after the varous models? And snce ths s OLS, we look at the squared resduals. SPSS outputs the "Sum of Squares" for the Resduals n the box labeled "ANOVA". We compare the sum of squares from the two models and see how much t has gone down wth the extra varables. A bg decrease ndcates that the new

11 varables are dong good work. And how do we know, how bg s "bg"? Compare t to some gven dstrbuton, n ths case the F dstrbuton. Bascally we look at the percent change n the sum of squares, so somethng lke: SSR0 SSR1 F SSR1 wth the wavy equals sgn to show that we're not qute done. Note that model 0 s the orgnal model and model 1 s the model wth the addtonal regressors, whch wll have a smaller resdual (so ths F can never be negatve). To make ths equal, we need to make t a bt lke an elastcty what s the percent change n the number of varables n the model? Suppose that we have N observatons and that the orgnal model has K varables, to whch we're consderng addng Q more observatons. Then the orgnal model has (N K 1) degrees of freedom [that "1" s for the constant term] whle the new model has (N K Q 1) degrees of freedom, so the Q dfference s Q. So the percent change n degrees of freedom s. Then the full N K Q 1 formula for the F test s SSR0 SSR 1 SSR1 F. Q N K Q 1 Whch s, admttedly, fugly. But we know ts dstrbuton, t's F wth (Q, N-K-Q-1) degrees of freedom the F-dstrbuton has 2 sets of degrees of freedom. Calculate that F, then use Excel to calculate FDIST(F,Q,N-K-Q-1), whch wll output a p-value for the test. If the p-value s less than 5%, reject the null hypothess. Multple Dummy Varables Multple dummy varables, D 1,, D 2,,,D J,, operate on the same basc prncple. Of course we can then have many further nteractons! Suppose we have dummes for educaton and mmgrant status. The coeffcent on educaton would tell us how the typcal person (whether mmgrant or natve) fares, whle the coeffcent on mmgrant would tell us how the typcal mmgrant (whatever her educaton) fares. An nteracton of more than Bachelor s degree wth Immgrant would tell how the typcal hghly-educated mmgrant would do beyond how the typcal mmgrant and typcal hghly-educated person would do (whch mght be dfferent, for both ends of the educaton scale). Many, Many Dummy Varables Don't let the name fool you you'd have to be a dummy not to use lots of dummy varables. For example regressons to explan people's wages mght use dummy varables for the ndustry n whch a person works. Regressons about fnancal data such as stock prces mght nclude dummes for the days of the week and months of the year.

12 Dummes for ndustres are often denoted wth labels lke "two-dgt" or "three-dgt" or smlar jargon. To understand ths, you need to understand how the government classfes ndustres. A specfc ndustry mght get a 4-dgt code where each dgt makes a further more detaled classfcaton. The frst dgt refers to the broad secton of the economy, as goods pass from the frst producers (farmers and mners, frst dgt zero) to manufacturers (1 n the frst dgt for non-durable manufacturers such as meat processng, 2 for durable manufacturng, 3 for hghertech goods) to transportaton, communcatons and utltes (4), to wholesale trade (5) then retal (6). The 7's begn wth FIRE (Fnance, Insurance, and Real Estate) then servces n the later 7 and early 8 dgts whle the 9 s for governments. The second and thrd dgts gve more detal: e.g. 377 s for sawmlls, 378 for plywood and engneered wood, 379 for prefabrcated wood homes. Some data sets mght gve you 5-dgt or even 6-dgt nformaton. These classfcatons date back to the 1930s and 1940s so some parts show ther age: the everncreasng number of computer parts go where plan "offce supples" used to be. The CPS data dstngushes between "major ndustres" wth 16 categores and "detaled ndustry" wth about 50. Creatng 50 dummy varables could be tresome so I recommend that you use SPSS's syntax edtor that makes cut-and-paste work easer. For example use the buttons to "compute" the frst dummy varable then "Paste Syntax" to see the general form. Then copy-and-paste and change the number for the 51 varables: COMPUTE d_nd1 = (a_dtnd EQ 1). COMPUTE d_nd2 = (a_dtnd EQ 2). COMPUTE d_nd3 = (a_dtnd EQ 3). COMPUTE d_nd4 = (a_dtnd EQ 4). COMPUTE d_nd5 = (a_dtnd EQ 5). COMPUTE d_nd6 = (a_dtnd EQ 6). COMPUTE d_nd7 = (a_dtnd EQ 7). You get the dea take ths up to 51. Then add them to your regresson! In other models such as predctons of sales, the specfcaton mght nclude a tme trend (as dscussed earler) plus dummy varables for days of the week or months of the year, to represent the typcal sales for, say, "a Monday n June". If you're lazy lke me, you mght not want to do all of ths mousework. (And f you really have a lot of varables, then you don't even have to be lazy.) There must be an easer way! There s. SPSS s a graphcal nterface that bascally wrtes SPSS code, whch s then submtted to the program. Clckng the buttons s wrtng computer code. Look agan at ths screen, where I've started codng the next dummy varable, ed_hs (from Transform\Compute Varables )

13 That lttle button, "Paste," can be a lot of help. It pastes the SPSS code that you just created wth buttons nto the SPSS Syntax Edtor.

14 Why s ths helpful? Because you can copy and paste these lnes of code, f you are only gong to make small changes to create a bunch of new varables. So, for example, the educaton dummes could be created wth ths code: COMPUTE ed_hs = (A_HGA = 39). VARIABLE LABELS ed_hs 'Hgh School Dploma'. COMPUTE ed_smc = (A_HGA > 39) & (A_HGA < 43). VARIABLE LABELS ed_smc 'Some College'. COMPUTE ed_coll = (A_HGA = 43). VARIABLE LABELS ed_coll 'College 4 Year Degree'. COMPUTE ed_adv = (A_HGA > 43). VARIABLE LABELS ed_adv 'Advanced Degree'. EXECUTE. Then choose "Run\All" from the drop-down menus to have SPSS execute the code. You can really see the tme-savng element f, for example, you want to create dummes for geographcal area. There s a code, GEDIV, that tells what secton of the country the respondent lves n. Agan these numbers have absolutely no nherent value, they're just codes from 1, New England, to 9, Pacfc regon. We can't put GEDIV nto a regresson but we can put geographc dummes. So we use the same procedure to create these: COMPUTE geo_1 = (GEDIV = 1). COMPUTE geo_2 = (GEDIV = 2). COMPUTE geo_3 = (GEDIV = 3). COMPUTE geo_4 = (GEDIV = 4). COMPUTE geo_5 = (GEDIV = 5). COMPUTE geo_6 = (GEDIV = 6). COMPUTE geo_7 = (GEDIV = 7). COMPUTE geo_8 = (GEDIV = 8). COMPUTE geo_9 = (GEDIV = 9). EXECUTE. You can begn to realze the tme-savng capablty here. Later we mght create 50 detaled ndustry and 25 detaled occupaton dummes. If at some pont you get stuck (maybe the "Run" returns errors) or f you don't know the syntax to create a varable, you can go back to the button-pushng dalogue box. The fnal advantage s that, f you want to do the same commands on a dfferent dataset (say, the March 2009) then as long as you have saved the syntax you can easly submt t agan. Wth enough dummy varables we can start to create some respectable regressons! Use "Data\Select Cases " to use only those wth a non-zero wage. Then do a regresson of wage on Age, race & ethncty (create some dummy varables for these), educatonal attanment, and geographc regon.

15 Why am I makng you do all of ths? Because I want you to realze all of the choces that go nto creatng a regresson or dong just about anythng wth data. There are a host of choces avalable to you. Some choces are rather conventonal (for example, the educaton breakdown I used above) but you need to know the feld n order to know what assumptons are common. Sometmes these commonplace assumptons conceal mportant nformaton. You want to do enough expermentaton to understand whch of your choces are crucal to your results. Then you can begn to understand how people mght analyze the exact same data but come to varyng conclusons. If your results contradct someone else's, then you have to fgure out what are the mportant assumptons that create the dfference.