Parametrisations, splines

Transcription

1 / 7 Parametrisations, splines Analysis of variance and regression course Marc Andersen, mja@statgroup.dk Analysis of variance and regression for health researchers, December, 20

2 2 / 7 Outline Parameters Definition Re-parametrisation Comparison of parameterisations Contrasts Model comparison Linear splines Dose-response models

3 Acknowledgements written by Lene Theil Skovgaard 2006, 2007 updated by Julie Lyng Forman 2008, November 2009 updated by Marc Andersen 2 April 2009, April 200, November 200, April 20, November 20 Dept. of Biostatistics 2 StatGroup 3 / 7

4 4 / 7 Parameter definition Parameter unknown quantity that we want to estimate (to estimate means to provide a good guess) the decrease in blood pressure following treatment the difference in decrease for treatment and placebo the increase in insulin growth factor (IGF-) with age (nuisance) parameter needed for technical reasons to complete the model, e.g. the residual variance in ANOVA part of a statistical model Parametrisation choice of which parameters are to enter the model description Re-parametrisation shift to a new set of parameters

5 / 7 Why choose a specific parametrisation? Ease - the program has some default parametrisations Test of specific hypotheses - difference between treatment and placebo - difference in height for boys and girls at the age of 4 Estimation of specific quantities: - the potency of a drug, ED 0 or ED 90 (explained later)

6 6 / 7 Example of re-parametrisations Change of scale/units Do we measure height in cm or m? Take the relation of lung capacity versus height: fev = α + β height If we change from measuring height in cm to m the regression coefficient (the parameter) changes from β to β = 00 β Change of origin/intercept choice of another reference group in ANOVA subtracting e.g. 70cm from all height measurements

7 7 / 7 Re-parametrisation Re-parametrisations do not change the model as such! same fitted values same normal regions and prediction limits but a possibility for interpretations of specific interest

8 8 / 7 Other re-parametrisations Re-parametrisation is applied in the more advanced situations (beyond linearity): non-linear regression, logistic regression, correlated observations Reasons for re-parametrisation knowledge of distributional assumptions Some parameter estimates may be more close to the normal distribution than others we like to be able to construct symmetric confidence intervals, using the standard error Avoid negative lower confidence limit for a positive parameter Linear models In linear models the estimates have exact normal distributions (provided the model assumptions are correct, of course...)

9 9 / 7 Example: Reumatoid Arthritis Population: 4 patients with Reumatoid Arthritis Treatment: Randomised to one out of 6 possible treatments: Placebo Aspirin (20 mg) One of 4 doses of an active anti-inflammatory drug, X (dose: 0 mg, mg, 20 mg, 2 mg) Outcome: An index index summing up the effectiveness of the treatment (decrease in various symptoms), the higher the better

10 0 / 7 Outcome: index-values Reference: Woolson, R.F. & Clarke, W.R.: Statistical methods for the analysis of biomedical data. 2ed., Wiley, (Exercise 0.4 page 409)

11 / 7 Representation of RA data in SAS Obs group type dose index placebo placebo placebo placebo placebo placebo placebo placebo placebo placebo placebo placebo placebo placebo x20 active x20 active x20 active x2 active x2 active x2 active x2 active x2 active 2 4.7

12 RA data: 4 active X-groups only 2 / 7

13 3 / 7 Two different parametrisations in One-way ANOVA Reference level supplemented with differences to reference level One level for the reference group (in SAS by default the last, numerically or alphabetically), supplemented with differences in levels from this reference group to each of the remaining groups good for describing diffences between groups Model: Y gi = α + β g + ε gi,β last = 0 One level for each group good for describing the individual groups Model: Y gi = µ g + ε gi

14 4 / 7 Traditional One-way ANOVA in SAS proc glm data=drug; where type= active ; class group; model index=group / solution; run; which yields the estimates: Standard Parameter Estimate Error t Value Pr > t Intercept B <.000 group x B <.000 group x B <.000 group x B <.000 group x B... NOTE: The X X matrix has been found to be singular...

15 / 7 Test that group levels are the same The GLM Procedure Dependent Variable: index Sum of Source DF Squares Mean Square F Value Pr > F Model <.000 Error Corrected Total R-Square Coeff Var Root MSE index Mean Source DF Type I SS Mean Square F Value Pr > F group <.000 Source DF Type III SS Mean Square F Value Pr > F group <.000 The test corresponds to all differences equal to zero

16 6 / 7 Details on the ANOVA-parametrisation in SAS Model written as a multiple regression: Y = β 0 + β x + β 2 x 2 + β 3 x 3 + ǫ The x s are so-called dummy variables or design variables: x is if subject i belongs to the first group, and 0 otherwise x 2 is if subject i belongs to the second group, and 0 otherwise x 3 is if subject i belongs to the third group, and 0 otherwise With this parametrisation, β 0 corresponds to the level for the last group (the reference group, here group 4); β is the difference in level between group and group 4 β 2 is the difference in level between group 2 and group 4,...

17 7 / 7 Parametrised with one level for each group No intercept Thus no reference group: Each group level is estimated. proc glm data=drug; where type= active ; class group; model index=group / noint solution; run; Standard Parameter Estimate Error t Value Pr > t group x <.000 group x <.000 group x <.000 group x <.000

18 8 / 7 Parametrised with one level for each group, continued All the tests now refer to the hypothesis of a zero level (which is not interesting). Please note that DF of group equals 4! Dependent Variable: index Sum of Source DF Squares Mean Square F Value Pr > F Model <.000 Error Uncorrected Total R-Square Coeff Var Root MSE index Mean Source DF Type I SS Mean Square F Value Pr > F group <.000 Source DF Type III SS Mean Square F Value Pr > F group <.000

19 9 / 7 Summary: Two parametrisations in One-way ANOVA One level, β 0 = µ 4, for the reference group supplemented with differences in level from the reference group to each of the remaining groups, β,β 2, and β 3 : β g = µ g µ 4 so that β g = 0 iff µ g = µ 4 good for testing of identity and certain pairwise comparisons One level, µ g, for each group good for estimation, not suited for testing!!

20 20 / 7 Contrasts If we want to compare dose 0 with dose : Test the hypothesis H : µ = µ 2, or equivalently H : µ 2 µ = 0 i.e. test that a contrast equals zero, namely µ 2 µ = µ + µ µ µ 4 The coefficients of the contrast are -,, 0, and 0

21 2 / 7 Estimate statements in GLM To compute the contrast and test that it is equal to zero: proc glm data=drug; where type= active ; class group; model index=group / noint solution; estimate dose mg vs. dose 0 mg group - 0 0; run; from which we get the additional output Standard Parameter Estimate Error t Value Pr > t dose mg vs. dose 0 mg <.000

22 22 / 7 Checking for variance heterogeneity Note: We have disregarded the problem of variance heterogeneity: proc glm data=drug; where type= active ; class group; model index=group / noint solution; means group / hovtest=levene; run; Levene s Test for Homogeneity of index Variance ANOVA of Squared Deviations from Group Means Sum of Mean Source DF Squares Square F Value Pr > F group Error A clear indication that the variance increases with dose. In practice: Deal with this! (Welch s test, log-transform...)

23 23 / 7 RA Example continued: 4 X-doses with linear regression line Can we use a simple model with linear dose effect?

24 24 / 7 Linear regression We look at the simple linear regression model: index = α + β dose + ε α is the intercept and β is the slope of the line, ε is the vertical distance from observation to line (the residual) SAS Program statements: proc reg data=drug; where type= active ; model index=dose / clb; run;

25 2 / 7 Linear regression results Sum of Mean Source DF Squares Square F Value Pr > F Model <.000 Error Corrected Total Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > t Intercept <.000 dose <.000 Conclusion: index increases approx. 2.8 points with each unit of dose in the range 0 2, if the model is correct...

26 26 / 7 Continued analysis Is this a reasonable description? Can we test the linearity? E.g. Is this model almost as good as a quadratic model the ANOVA model (Still ignoring differences in variances)

27 27 / 7 Quadratic fit: include new variable dose2=dose 2 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > t Intercept dose <.000 dose

28 28 / 7 Model reduction: group means to linear F test for comparison of explained variation We have to compare two models: Model The ANOVA model with 4 separate groups Model 2 The linear regression model with actual dose as covariate Note: The models have to be nested, i.e. one is derived from the other, typically be demanding some parameters to be equal to each other or equal to zero Indeed, the linear regression model is a special case of the ANOVA model, in which certain contrast are equal.

29 Model comparison: Sums of squares Model Sum of Squares(Model): i (ŷ i ȳ) 2 Explained variation: How much do the predicted values vary? (the bigger the better) Residuals Sum of Squares(Residual): i (y i ŷ i ) 2 Unexplained variation (residual variation): How large are the deviations from the model? (the smaller the better) Total Sum Squares(Total): i (y i ȳ) 2 Total variation from overall mean. The Model and Residual variation partitions the total variation. (this is fixed) Notation: y i : Observed, ŷ i : Predicted, y i ŷ i : Residual, ȳ: overall mean. 29 / 7

30 30 / 7 Comparing models How Look at changes in the explained variation Sum of Squares for the model (ie where source is not error) Sum Sq = Sum Sq(Model ) Sum Sq(Model 2 ) > 0, by defintion as more parameters can always explain more of the variation How much less is explained by the simpler model? Sum Sq(Model 2 ) Sum Sq(Residuals 2 ) { Model 2 }} { { Residual 2 }} { Sum Sq } {{ } } {{ } Model Residual Sum Sq(Model ) Sum Sq(Residuals )

31 3 / 7 Comparing models details df = df(model ) df(model 2 ) Mean Sq = Sum Sq/ df F = Mean Sq Mean Sq(Residual) How large can this get, just by chance/coincidence? F F(df 2 df, df ) where df and df 2 denote the degrees of freedom for the Mean Square(Residual) for the two models. Note: df(model ) + df = N, df(model 2 ) + df 2 = N.

32 32 / 7 Comparing models using F-test df SS MS Model - Groups Model 2 - Linear Change Model Residual F = = , F F(2, 2) corresponds to a P-value of If we have many groups, the test is not very powerful Note: The excessive number of digits are only used for being able to trace and re-produce the calculations. Usually the numbers should be shown with fewer digits.

33 33 / 7 SAS Type I and Type III sums of squares Type I Type III : sum of squares obtained for the effect in question while keeping the preceding effects (sequential) : sum of squares obtained when removing the effect and keeping all other effects Type I sum of squares depends on the order the terms in the model are specified. Often used to document a sequence of model fit. Type III sum of squares are used to identify effects, that can be removed from the model.

34 34 / 7 Comparing models using SAS The usual way of testing linearity: Trick SAS into doing the test by including dose both as a linear effect and as a class effect: proc glm data=drug; where type= active ; class group; model index=dose group / solution; run; The term group represents the variation of dose group means around the straight line

35 3 / 7 SAS output, Sum of Squares and Type I analysis Dependent Variable: index Sum of Source DF Squares Mean Square F Value Pr > F Model <.000 Error Corrected Total Source DF Type I SS Mean Square F Value Pr > F dose <.000 group

36 36 / 7 SAS output, Type III analysis and parameter estimates Source DF Type III SS Mean Square F Value Pr > F dose group Standard Parameter Estimate Error t Value Pr > t Intercept B dose B <.000 group x B group x B group x B... group x B... Note: The effect group now has only 2 degrees of freedom, since we are not testing equality, but linearity! Note: Type III test of dose is not meaningful here and the parameters do not have a simple interpretation.

37 37 / 7 Identifying linearity using contrasts Relation between linear regression and ANOVA model If the linear regression model is true, then all the contrasts µ 2 µ, µ 3 µ 2, and µ 4 µ 3 are equal to β as the doses are 0,, 20, 2. Derivation of coefficients contrasts µ 2 µ = µ 3 µ 2 if and only if (µ 3 µ 2 ) (µ 2 µ ) = µ 2µ 2 + µ 3 = 0 µ 3 µ 2 = µ 4 µ 3 if and only if (µ 4 µ 3 ) (µ 3 µ 2 ) = µ 2 2µ 3 + µ 4 = 0

38 38 / 7 Contrast statement in GLM Contrast statement in SAS proc glm data=drug; where type= active ; class group; model index=group / noint solution; contrast dev linearity group -2 0, group 0-2 ; run; SAS output Contrast DF Contrast SS Mean Square F Value Pr > F dev linearity

39 39 / 7 Unequally spaced doses The previous simple expression was due to equally spaced doses In case doses were 0, 7, 9, and 2: µ 2 µ = 7 β µ 3 µ 2 = 2 β µ 4 µ 3 = 6 β Hence test linearity by testing that µ 2 µ 7 = µ 3 µ 2 2 = µ 4 µ 3 6

40 40 / 7 Example: Growth Relate weight to age and gender, in the Juul-data for Tanner stage Again: Problems with variance homogeneity...

41 4 / 7 Two different parametrisations in ANCOVA One regression line for the reference group (in SAS the last, numerically or alphabetically), supplemented with the differences in intercept and slope from this reference group to the remaining group good for describing the diffence between groups One regression line for each group good for describing the evolution in the individual groups Analysis of covariance, ANCOVA (model with one Class variable and one quantitative variable.)

42 42 / 7 Test that regression lines are parallel Source DF Type I SS Mean Square F Value Pr > F sex <.000 age <.000 age*sex Source DF Type III SS Mean Square F Value Pr > F sex age <.000 age*sex

43 Type I and Type III analysis The model can be described as weigth = α sex + β sex age + ǫ Type I analysis is sequential testing (from below) of: H 0 : β boy β girl = 0, i.e. slopes are the same H : β = 0, i.e. no age-effect H 2 : α boy α girl = 0, i.e. no sex-effect either Type III analysis is non-sequential: H 0 : β boy β girl = 0, i.e. slopes are the same H 0 : β boy = 0, i.e. slope of reference is zero H 0 : α boy α girl = 0, i.e. intercepts are the same slopes may differ 43 / 7

44 44 / 7 Parameter estimates Standard Parameter Estimate Error t Value Pr > t Intercept B sex female B sex male B... age B <.000 age*sex female B age*sex male B... NOTE: The X X matrix has been found to be singular, and a generalized inverse was used to solve the normal equations. Terms whose estimates are followed by the letter B are not uniquely estimable. Interpretation Boys increase their weight more rapidly than girls: approx. 692 g. more per year.

45 4 / 7 SAS-computation of both lines simultaneously Subgroup analysis Keep the interaction term sex*age Leave out the marginal effect age this will merge the marginal effect into the interaction term Leave out the intercept (use option noint) this will merge the intercept into the sex effect proc glm data=juul; where tanner=; class sex; model weight=sex age*sex / noint solution clparm; run;

46 46 / 7 SAS-computation of both lines simultaneously The GLM Procedure Dependent Variable: weight Sum of Source DF Squares Mean Square F Value Pr > F Model <.000 Error Uncorrected Total R-Square Coeff Var Root MSE weight Mean Source DF Type I SS Mean Square F Value Pr > F sex <.000 age*sex <.000

47 47 / 7 SAS-computation of both lines simultaneously, estimates Source DF Type III SS Mean Square F Value Pr > F sex <.000 age*sex <.000 Standard Parameter Estimate Error t Value Pr > t sex female <.000 sex male age*sex female <.000 age*sex male <.000 Parameter 9% Confidence Limits sex female sex male age*sex female age*sex male

48 48 / 7 Simultaneous test of both sex-effects SAS code proc glm data=juul; where tanner=; class sex; model weight=sex age*sex / noint solution clparm; contrast sex and sex*age sex -, age*sex - ; run; SAS output Contrast DF Contrast SS Mean Square F Value Pr > F sex and sex*age

49 49 / 7 Estimate the expected weight difference at age 4 years SAS code proc glm data=juul; where tanner=; class sex; model weight=sex age*sex / noint solution clparm; estimate difference at 4 sex - age*sex -4 4; run; SAS output Standard Parameter Estimate Error t Value Pr > t difference at Parameter 9% Confidence Limits difference at

50 0 / 7 Summary: Same model, 2 different parametrisations proc glm data=juul; where tanner=; class sex; model weight=sex age age*sex / solution; run; An intercept for the reference group (sex= male ) A difference from sex= female to sex= male An effect of age (slope) for the reference group A difference in slopes from sex= female to sex= male proc glm data=juul; where tanner=; class sex; model weight=sex age*sex / noint solution; run; An intercept for each group (sex) An effect of age (slope) for each group (sex)

51 / 7 Reumatioid Arthritis example including placebo Now we include the Placebo group as dose = 0. Does the Placebo treatment fit in here? No, obviously not. Either we have a placebo effect or a threshold effect. But how could we make a formal test?

52 2 / 7 Model with placebo-effect Including placebo effect in model index = α + β dose + γ I(placebo) + ε gi γ is the deviation of the placebo group from the regression line I(placebo) is the indicator of the placebo-group, is the treatment is placebo, 0 otherwise. Test H 0 : γ = 0, the placebo group is in line with the other doses

53 3 / 7 Model with placebo-effect, SAS code and output SAS code data drug; set drug; active_drug=(dose>0); run; proc glm data=drug; where group ne aspirin ; class active_drug; model index= dose active_drug / solution; run; SAS output Source DF Type I SS Mean Square F Value Pr > F dose <.000 active_drug <.000

54 4 / 7 Model with placebo-effect, parameter estimates SAS output Standard Parameter Estimate Error t Value Pr > t Intercept B <.000 dose <.000 active_drug B <.000 active_drug B... NOTE: The X X matrix has been found to be singular, and a generalized inverse was used to solve the normal equations. Terms whose estimates are followed by the letter B are not uniquely estimable. Interpretation The placebo group (active_drug=0) lies an estimated above the expected 0-dose level.

55 Example: Juul-data, Serum IGF- for boys We have seen different age dependencies in separate Tanner stages. Is this simply due to a nonlinear age effect? / 7

56 6 / 7 Modelling non-linear effects Polynomials tend to be very wiggly (use i=rq or i=rc in the symbol-statement) Splines: piecewise interpolations, often linear or cubic

57 7 / 7 Separate lines for each age group Approach Subdivide age into groups, using appropriate thresholds Fit linear effect of age in each age group The result is unconnected lines:

58 8 / 7 Linear splines Approach Subdivide age into groups, using appropriate thresholds Fit linear effect of age in each age group Make the linear pieces meet at the thresholds The result is a bended line, a so-called linear spline:

59 Linear spline explained Using threshold ages 0, 2, 3, and years ssigf = δ 0 + δ age + δ 2 (age 0) + + δ 3 (age 2) + + δ 4 (age 3) + + δ (age ) + where (age x) + = age x if age > x and zero otherwise the line for age 0 has slope β = δ the line for age 0 2 has slope β 2 = δ + δ 2 the line for age 2 3 has slope β 3 = δ + δ 2 + δ 3 the line for age 3 has slope β 4 = δ + δ 2 + δ 3 + δ 4 the line for age 20 has slope β = δ + δ 2 + δ 3 + δ 4 + δ Parameters represent changes in slope δ 2 = β 2 β, δ 3 = β 3 β 2,... 9 / 7

60 60 / 7 Linear spline, SAS code data juul; set juul; /* define new intercept value ssigf-level at age ) */ age=age-; /* define number of years above certain threshold ages */ extra_age0=max(age-0,0); extra_age2=max(age-2,0); extra_age3=max(age-3,0); extra_age=max(age-,0); run; /* fit splines for boys and girls separately */ proc sort data=juul; by sex; run; proc reg data=juul; where age ge and age le 20; by sex; model ssigf=age extra_age0 extra_age2 extra_age3 extra_age; run;

61 6 / 7 Derived threshold variables age_ extra_ extra_ extra_ extra_ Obs age group age0 age2 age3 age

62 62 / 7 Fitting model with threshold variables Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model <.000 Error Corrected Total Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > t Intercept <.000 age extra_age extra_age extra_age extra_age <.000

63 63 / 7 Can we reduce the model to a simpler one? Quadratic age effect... not possible - why not? To simple linearity... not reasonable: proc glm data=juul; where age ge and age le 20; by sex; model ssigf=age extra_age0 extra_age2 extra_age3 extra_age / solution; contrast all extra_age0, extra_age2, extra_age3, extra_age ; run; Contrast DF Contrast SS Mean Square F Value Pr > F all <.000

64 64 / 7 Test of adequacy of linear spline? Test against a more complicated model separate regressions for each age group Inclusion of tanner as a Class-variable... Don t forget to check the residual plots!

65 6 / 7 Dose-response curves Example of a typical dose-response relation, for moderate doses We have almost linearity in this dose range (dose 22 38)

66 66 / 7 Dose-response curves full range For extreme doses we see a clear deviation from linearity and: smaller variation in the ends

67 67 / 7 Theoretical dose response relation This looks like sigmoid shape S-shaped, increasing from 0 to 00 % example: a logistic curve response = 00 +γ exp( β log(dose)) Note: the logit transformation is: logit(response) = log ( response 00 response )

68 68 / 7 Example from anaesthesia 47 patients to be operated with two different anesthetics Halothane Neurolept Y: Twitch response at the ulnar nerve (at the thumb), in % X: Dose of muscle relaxantia group=halothane patient dose depression group=neurolept patient dose depression

69 69 / 7 Anaesthesia data Halotan Neurolept

70 70 / 7 Dosis response curve proc gplot; plot logit*dose / haxis=axis vaxis=axis2 frame; axis logbase=2 logstyle=expand value=(h=2) minor=none label=(h=3 Dose on log scale ); axis2 value=(h=2) minor=none label=(a=90 R=0 H=3 Logit transformed response ); symbol v=circle i=sm60 h=3 c=black l= w=2; run; For the logistic dose response model we get linearity by transforming the data using the logit transformation: logit(response) = logγ + β log(dose)

71 7 / 7 Transformation to linearity in the logistic dose response model y x ( ) twitch : logit_twitch = log 00 twitch : logdose = log(dose) Assume linear relation: logit_twitch = α + βlogdose + ε where α = logγ

72 Neurolept: Logistic dose response model seems OK 72 / 7

73 73 / 7 Neurolept: Parameter estimates Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > t Intercept <.000 logdose <.000 Parameter Estimates Variable DF 9% Confidence Limits Intercept logdose

74 74 / 7 Deriving estimates for α and β We get estimates of α and β from the equation logit_twitch = α + β logdose But: What about the parameters of interest, i.e. ED 0 and ED 90? From α + β log(ed 0 ) = logit(0) = 0 we get ˆ ED 0 = exp( ˆα/ˆβ) How do we calculate s.e.( ˆ ED 0 )? Re-parametrisation: γ = log(ed 0 ) = α/β γ 2 = log(ed 90 ) = (logit(90) α)/β

75 7 / 7 The model may then be written as: y= logit_twitch = logit(90) x γ γ 2 γ = 2.97 x γ γ 2 γ This function is nonlinear in γ and γ 2! Direct estimation of γ and γ 2 using non-linear regression... more about this in subsequent lectures