LINEAR REGRESSION ANALYSIS MODULE XII Lecture - 6 Polynomal Regresson Models Dr. Shalabh Department of Mathematcs and Statstcs Indan Insttute of Technology Kanpur
Test of sgnfcance To test the sgnfcance of hghest order term, we test the null hypothess H :. α k = Ths hypothess s equvalent to We would use H : β k = n polynomal regresson model. F = reg ( α ) SS ( k) / ( n k ) res SS k n ˆ αk Pk( x) y = = SS ( k) / ( n k ) res ~ F(, n k ) under H. If order of the model s changed to ( k r), we need to compute only r new coeffcents. The remanng coeffcents ˆ α, ˆ α,..., ˆ αk do not change due to orthogonalty property of polynomals. Thus the sequental fttng of the model s computatonally easy. When X are equally spaced, the tables of orthogonal polynomals are avalable and the orthogonal polynomals can be easly constructed.
Frst 7 orthogonal polynomals are as follows: Let d be the spacng between levels of x and values. The tables are avalable. { λ j } be the constants chosen so that polynomals wll have nteger P( x ) = x x P( x ) = λ d ( x x) n P( x ) = λ d ( x x) x x n 7 P( x ) = λ d d 4 ( x x) x x n ( n )( n 9) P4( x ) = λ4 d d 4 56 5 ( x x) 5 x x 4 x x P5( x ) = λ5 ( n 7) ( 5n n 47) ) d 8 d 8 d ( x x) 5 x x 4 x x P6( x ) = λ6 ( n ) ( 5n n 9) ) d 44 d 76 d 6 4 5 ( n )( n 4784 9)( n 5)
An example of the table for n = 5 s as follows: 4 n x P P P P 4 - - - - - -4 5 { P ( )} j x 4 7 = λ The orthogonal polynomals can also be constructed when x s are not equally spaced. Pecewse polynomal (Splnes) Sometmes t s exhbted n the data that a lower order polynomal does not provde a good ft. A possble soluton n such stuaton s to ncrease the order of the polynomal but t may always not work. The hgher order polynomal may not mprove the ft sgnfcantly. Such stuatons can be analyzed through resduals, e.g., the resdual sum of square may not stablze or the resdual plots fal to explan the unexplaned structure. One possble reason for such happenng s that the response functon has dfferent behavor n dfferent ranges of ndependent varables. Ths type of problems can be overcome by fttng an approprate functon n dfferent ranges of explanatory varable. So polynomal wll be ftted nto peces. The splne functon can be used for such fttng of polynomal n peces. 5 6 5
Splnes and knots The pecewse polynomals are called splnes. The jont ponts of such peces are called as knots. If polynomal s of order k, then the splne s a contnuous functon wth (k - ) contnuous dervatves. For ths, the functon values and frst (k - ) dervatves agree at the knots. 5 Cubc splnes: For example, consder a cubc splne wth h knots. Suppose the knots are contnuous frst and second dervatves at these knots. Ths can be expressed as t < t <... < th and cubc splne has where h j oj j= = Ey ( ) = Sx ( ) = β x β ( x t) x t f x t > ( x t ) = f x t. It s assumed that the poston of knots are known. Under ths assumpton, ths model can be ftted usng the usual fttng methods of regresson analyss lke least squares prncpal. In case, the knot postons are unknown, then they can be consdered as unknown parameters whch can be estmated. But n such stuaton, the model becomes non-lnear and methods of non-lnear regresson can be used.
Issue of number and poston of knots It s not so smple to know the number and poston of knots n a gven set of data. It s tred to keep the number of knots as mnmum as possble and each segment should have mnmum four or fve data ponts. There should not be more than one extreme pont and one pont of nflexon n each segment. If such ponts are to be accommodated, then t s suggested to keep the extreme pont n the center of segment and pont of nflexon near the knots. It s also possble to ft the polynomals of dfferent orders n each segment and to mpose dfferent contnuty restrctons at the knots. Suppose t s to be accomplshed n a cubc splne model. If all cubc splne model wthout contnuty restrctons s ( h ) peces of polynomal are cubc, then a 6 where h j oj j= = j= j Ey ( ) = Sx ( ) = β x β ( x t) f x > ( x t ) = f x. If the term β ( ) j j x t s n the model, then j th dervatve of S (x) at t s dscontnuous. If the term ( x t ) j s not n the model, then j th dervatve of S (x) s contnuous at t. β j So the model s ftted better when requred contnuty restrctons are fewer because then more parameters wll be ncluded n the model. If more contnuty restrctons are needed, then t ndcates that the model s not well ftted but the fnally ftted curve wll be smoother. The test of hypothess n multple regresson model can be used to determne the order of polynomal segments and contnuty restrctons.
Example Suppose there s only one knot at t n a cubc splne wthout contnuty restrctons gven by 7 E( y) = S( x) = β β x β x β x β ( x t) β ( x t) β ( x t) β ( x t). The term nvolvng β and are present n the model, so, ts frst dervatve and second dervatve, β β Sx ( ) S'( x) are not necessarly contnuous at t. Next queston arses s how to judge the qualty of ft. Ths can be done by S"( x) test of hypothess as follows: H : β = tests the contnuty of Sx ( ) H : β = β = H : β = β = β = tests the contnuty of Sx ( ) and S'( x) tests the contnuty of Sx ( ), S'( x ) and S"( x). The test H : β = β = β = β = ndcates that cubc splne fts data better than a sngle cubc polynomal over the range of explanatory varable x. Ths approach s not satsfactory f the knots are large n number as ths makes X X ll-condtoned. Ths problem s solved by usng cubc B- splne whch are defned as ( x tj ) B ( x) =,,,..., 4 = h j= 4 ( tj tm) m = 4 m j h 4 Ey ( ) = Sx ( ) = γ B( x) = where γ ' s ( =,,..., h 4) are parameters to be estmated. There are eght more knots - t < t < t < t and t. h < th < th < th 4 Choose t = xmn, th = xmax and other knots arbtrarly.
8 Polynomal models n two or more varables The technques of fttng of polynomal model n one varable can be extended to fttng of polynomal models n two or more varables. A second order polynomal s more used n practce and ts model s specfed by y= β β x β x β x β x β xx ε. Ths s also termed as response surface. The methodology of response surface s used to ft such models and helps n desgnng an experment.