Examples of fitting various piecewise-continuous functions to data, using basis functions in doing the regressions.

Size: px

Start display at page:

Download "Examples of fitting various piecewise-continuous functions to data, using basis functions in doing the regressions."

Alice Sheila Bishop
5 years ago
Views:

1 Examples of fitting various piecewise-continuous functions to data, using basis functions in doing the regressions. David. Boore These examples in this document used R to do the regression. See also Notes_on_piecewise_continuous_regression.doc for more detail on why the basis functions used below guarantee continuity at the breakpoints. Based on Eric Thompson's program piecewise-continuous.r, using DB basis functions CONSTRUCT DATA TO BE FIT: Define function QUADRATIC, LINEAR, LINEAR Breakpoints: c <- c(4, 7) y <- function(,c){ ifelse(<c[1], *(-c[1])-0.5*(-c[1])^2, ifelse(<=c[2], *(-c[1]), *(c[2]-c[1])-0.5*(-c[2]))) } Generate data set.seed(1) n <- 300 <- runif(n, 0, 10) Add some noise: yn <- y(,c) + rnorm(n, sd = 0.5) plot(, yn,col="black") abline(v = c) lines(sort(),y(sort(),c),lwd=3,col="red") 1

2 yn Compute basis function for each region: b1 <- function(x,r){ifelse(x<r,x,r)} b1.2 <- function(x,r){ifelse(x<r,x^2,r^2)} b2 <- function(x,l,r){ifelse(x<l,0,ifelse(x<r,x-l,r-l))} b3 <- function(x,l){ifelse(x<l,0,x-l)} plot(, b1(,c[1]), ylim=c(-0.25,10.0),col = "blue", pch = 20) points(, b1.2(,c[1]), col = "red", pch = 20) points(, b2(,c[1],c[2]), col = "green", pch = 20) points(, b3(,c[2]), col = "purple", pch = 20) 2

3 abline(v = c) b1(, c[1]) odel <- lm(yn ~ b1(,c[1]) + b1.2(,c[1]) + b2(,c[1],c[2]) + b3(,c[2]) ) Regression summary: summary(odel) Call: lm(formula = yn ~ b1(, c[1]) + b1.2(, c[1]) + b2(, c[1], c[2]) + b3(, c[2])) Residuals: in 1Q edian 3Q ax

4 Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) <2e-16 *** b1(, c[1]) <2e-16 *** b1.2(, c[1]) <2e-16 *** b2(, c[1], c[2]) <2e-16 *** b3(, c[2]) <2e-16 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 295 degrees of freedom ultiple R-squared: , Adjusted R-squared: F-statistic: 3529 on 4 and 295 DF, p-value: < 2.2e-16Coefficients: coef <- coefficients(odel) coef 4

5 yn

6 FLAT, QUADRATIC, LINEAR, LINEAR Define function Breakpoints: c <- c(2, 4, 7) y <- function(,c){ ifelse(<c[1], -3.67, ifelse(<c[2], *(-c[2])-0.5*(-c[2])^2, ifelse(<=c[3], *(-c[2]), *(c[3]-c[2])-0.5*(-c[3])))) } Generate data set.seed(1) n <- 300 <- runif(n, 0, 10) Add some noise: yn <- y(,c) + rnorm(n, sd = 0.5) plot(, yn,col="black") abline(v = c) lines(sort(),y(sort(),c),lwd=3,col="red") 6

7 yn Compute basis function for each region: b1 <- function(x){rep(1,length(x))} Alternatively, but less desirable: b1 <- rep(1,length()) b2 <- function(x,l,r){ifelse(x<l,0,ifelse(x<r,x-l,r-l))} b2.2 <- function(x,l,r){ifelse(x<l,0,ifelse(x<r,(x-l)^2,(r-l)^2))} b3 <- function(x,l,r){ifelse(x<l,0,ifelse(x<r,x-l,r-l))} b4 <- function(x,l){ifelse(x<l,0,x-l)} plot(, b1(), ylim=c(-0.25,5.0),col = "blue", pch = 20) Alternatively, but less desirable: plot(, b1, ylim=c(-0.25,5.0),col = "blue", pch = 20) points(, b2(,c[1],c[2]), col = "red", pch = 20) 7

8 points(, b2.2(,c[1],c[2]), col = "magenta", pch = 20) points(, b3(,c[2],c[3]), col = "green", pch = 20) points(, b4(,c[3]), col = "purple", pch = 20) abline(v = c) b1() odel <- lm(yn ~ -1 + b1() + b2(,c[1],c[2]) + b2.2(,c[1],c[2]) + b3(,c[2],c[3]) + b4(,c[3]) ) Alternatively, but less desirable: odel <- lm(yn ~ 1 + b2(,c[1],c[2]) + b2.2(,c[1],c[2]) + b3(,c[2],c[3]) + b4(,c[3]) ) Regression summary: summary(odel) 8

9 Call: lm(formula = yn ~ -1 + b1() + b2(, c[1], c[2]) + b2.2(, c[1], c[2]) + b3(, c[2], c[3]) + b4(, c[3])) Residuals: in 1Q edian 3Q ax Coefficients: Estimate Std. Error t value Pr(> t ) b1() < 2e-16 *** b2(, c[1], c[2]) < 2e-16 *** b2.2(, c[1], c[2]) *** b3(, c[2], c[3]) < 2e-16 *** b4(, c[3]) < 2e-16 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 295 degrees of freedom ultiple R-squared: , Adjusted R-squared: F-statistic: 1069 on 5 and 295 DF, p-value: < 2.2e-16 coef <- coefficients(odel) coef Replot data plot(, yn,col="black") Plot actual function abline(v = c) lines(sort(),y(sort(),c),lwd=2,col="red") Add predictions p <- sort() yp <- coef[1]*b1(p)+coef[2]*b2(p,c[1],c[2])+coef[3]*b2.2(p,c[1],c[2]) + coef[4]*b3(p,c[2],c[3]) + coef[5]*b4(p,c[3]) Alternatively, but less desirable: yp <- coef[1]+coef[2]*b2(p,c[1],c[2])+coef[3]*b2.2(p,c[1],c[2]) + coef[4]*b3(p,c[2],c[3]) + coef[5]*b4(p,c[3]) lines(p,yp, lwd=2, col="blue") 9

10 yn

11 FLAT, QUADRATIC, LINEAR, FLAT Define function Breakpoints: c <- c(2, 4, 7) y <- function(,c){ ifelse(<c[1], -3.67, ifelse(<c[2], *(-c[2])-0.5*(-c[2])^2, ifelse(<=c[3], *(-c[2]), *(c[3]-c[2])))) } Generate data set.seed(1) n <- 300 <- runif(n, 0, 10) Add some noise: yn <- y(,c) + rnorm(n, sd = 0.5) plot(, yn,col="black") abline(v = c) lines(sort(),y(sort(),c),lwd=3,col="red") 11

12 yn Compute basis function for each region: b1 <- function(x){rep(1,length(x))} b2 <- function(x,l,r){ifelse(x<l,0,ifelse(x<r,x-l,r-l))} b2.2 <- function(x,l,r){ifelse(x<l,0,ifelse(x<r,(x-l)^2,(r-l)^2))} b3 <- function(x,l,r){ifelse(x<l,0,ifelse(x<r,x-l,r-l))} plot(, b1(), ylim=c(-0.25,5.0),col = "blue", pch = 20) points(, b2(,c[1],c[2]), col = "red", pch = 20) points(, b2.2(,c[1],c[2]), col = "magenta", pch = 20) points(, b3(,c[3],c[4]), col = "green", pch = 20) abline(v = c) 12

13 b odel <- lm(yn ~ -1 + b1() + b2(,c[1],c[2]) + b2.2(,c[1],c[2]) + b3(,c[2],c[3]) ) Regression summary: summary(odel) Call: lm(formula = yn ~ 1 + b2(, c[1], c[2]) + b2.2(, c[1], c[2]) + b3(, c[2], c[3])) Residuals: in 1Q edian 3Q ax

14 Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) < 2e-16 *** b2(, c[1], c[2]) < 2e-16 *** b2.2(, c[1], c[2]) e-06 *** b3(, c[2], c[3]) < 2e-16 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 296 degrees of freedom ultiple R-squared: , Adjusted R-squared: F-statistic: 2204 on 3 and 296 DF, p-value: < 2.2e-16 coef <- coefficients(odel) coef Replot data plot(, yn,col="black") Plot actual function abline(v = c) lines(sort(),y(sort(),c),lwd=2,col="red") Add predictions p <- sort() yp <- coef[1]*b1(p)+coef[2]*b2(p,c[1],c[2])+coef[3]*b2.2(p,c[1],c[2]) + coef[4]*b3(p,c[2],c[3]) lines(p,yp, lwd=2, col="blue") 14

15 yn

16 FLAT, QUADRATIC, FLAT, LINEAR Define function Breakpoints: c <- c(2, 4, 7) y <- function(,c){ ifelse(<c[1], -3.67, ifelse(<c[2], *(-c[2])-0.5*(-c[2])^2, ifelse(<=c[3],1.13, *(-c[3])))) } Generate data set.seed(1) n <- 300 <- runif(n, 0, 10) Add some noise: yn <- y(,c) + rnorm(n, sd = 0.5) plot(, yn,col="black") abline(v = c) lines(sort(),y(sort(),c),lwd=3,col="red") 16

17 yn Compute basis function for each region: b1 <- function(x){rep(1,length(x))} b2 <- function(x,l,r){ifelse(x<l,0,ifelse(x<r,x-l,r-l))} b2.2 <- function(x,l,r){ifelse(x<l,0,ifelse(x<r,(x-l)^2,(r-l)^2))} b4 <- function(x,l){ifelse(x<l,0,x-l)} plot(, b1(), ylim=c(-0.25,5.0),col = "blue", pch = 20) points(, b2(,c[1],c[2]), col = "red", pch = 20) points(, b2.2(,c[1],c[2]), col = "magenta", pch = 20) points(, b4(,c[3]), col = "purple", pch = 20) abline(v = c) 17

18 b odel <- lm(yn ~ -1 + b1() + b2(,c[1],c[2]) + b2.2(,c[1],c[2]) + b4(,c[3]) ) Regression summary: summary(odel) Call: lm(formula = yn ~ 1 + b2(, c[1], c[2]) + b2.2(, c[1], c[2]) + b4(, c[3])) Residuals: in 1Q edian 3Q ax

19 Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) < 2e-16 *** b2(, c[1], c[2]) < 2e-16 *** b2.2(, c[1], c[2]) e-07 *** b4(, c[3]) < 2e-16 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 296 degrees of freedom ultiple R-squared: , Adjusted R-squared: F-statistic: 1256 on 3 and 296 DF, p-value: < 2.2e-16 coef <- coefficients(odel) coef Replot data plot(, yn,col="black") Plot actual function abline(v = c) lines(sort(),y(sort(),c),lwd=2,col="red") Add predictions p <- sort() yp <- coef[1]*b1(p)+coef[2]*b2(p,c[1],c[2])+coef[3]*b2.2(p,c[1],c[2]) + coef[4]*b4(p,c[3]) lines(p,yp, lwd=2, col="blue") 19

20 yn

21 FLAT, LINE CROSSING 0.0 AT XC, FLAT How do a model that is flat to c[1], forced to cross 0.0 at x = xc, and then is flat beyond c[2]? Thinking about it, there is only one basis function, even though it has a break at x=c[1]. The reason is that the slope of the line between c[1] and c[2] is determined by the value of constant portion for x < c[1] and the condition that the line crosses the zero line at x=xc. Therefore the slope is NOT a regression parameter. There is only one regression parameter. FLAT, LINEAR GOING THROUGH A SPECIFIED POINT, FLAT Define function Breakpoints: c <- c(4, 6) cz <- 5.5 slope <- ( )/(cz-c[1]) y <- function(x,c){ ifelse(x<c[1], 2.0, ifelse(x<c[2],2.0 - slope*(x-c[1]), slope*(c[2]-c[1]) )) } Generate data set.seed(1) n <- 300 <- runif(n, 0, 10) Add some noise: yn <- y(,c) + rnorm(n, sd = 0.5) plot(, yn,col="black") abline(h = 0) abline(v = c(c[1],cz,c[2])) lines(sort(),y(sort(),c),lwd=3,col="red") 21

22 yn Compute basis function for each region: b1 <- function(x,l,xc,r){ifelse(x<l,1,ifelse(x<r,1-(x-l)/(xc-l),1-(r-l)/(xc- L)))} plot(, b1(,c[1],cz,c[2]), col = "blue", pch = 20) abline(h=0) abline(v = c(c[1],cz,c[2])) 22

23 b1(, c[1], cz, c[2]) odel <- lm(yn ~ -1 + b1(,c[1],cz,c[2]) ) Regression summary: summary(odel) Call: lm(formula = yn ~ -1 + b1(, c[1], cz, c[2])) Residuals: in 1Q edian 3Q ax Coefficients: 23

24 Estimate Std. Error t value Pr(> t ) b1(, c[1], cz, c[2]) <2e-16 *** --- Signif. codes: 0 *** ** 0.01 * Residual standard error: on 299 degrees of freedom ultiple R-squared: , Adjusted R-squared: F-statistic: 2486 on 1 and 299 DF, p-value: < 2.2e-16 coef <- coefficients(odel) coef b1(, c[1], cz, c[2]) Replot data plot(, yn,col="black") Plot actual function abline(h = 0) abline(v = c(c[1],cz,c[2])) lines(sort(),y(sort(),c),lwd=2,col="red") Add predictions p <- sort() yp <- coef[1]*b1(p,c[1],cz,c[2]) lines(p,yp, lwd=2, col="blue") 24

25 yn

cor(dataset$measurement1, dataset$measurement2, method= pearson ) cor.test(datavector1, datavector2, method= pearson )

cor(dataset$measurement1, dataset$measurement2, method= pearson ) cor.test(datavector1, datavector2, method= pearson ) Tutorial 7: Correlation and Regression Correlation Used to test whether two variables are linearly associated. A correlation coefficient (r) indicates the strength and direction of the association. A correlation