Chemometrcs Unt : Regresson Analyss The problem of predctng the average value of one varable n terms of the known values of other varables s the problem of regresson. In carryng out a regresson analyss t s necessary to ft data to some assumed equaton usng the method of least squares. However there s more to a regresson analyss than just dong a least squares ft and ths s explaned n what follows. In analytcal chemstry t s very common for the assumed equaton to be of the form y = a + bx.e. a lnear equaton wth two varables, x and y. For example y could be the absorbance of a solute n soluton and x the concentraton of the solute, (Beer's Law). In ths example the ntercept s, n prncple, zero, so that the equaton s y = bx. Before the advent of computers and cheap calculators t was customary to ft the data by plottng a (straght lne) graph of y versus x. Ths was the "calbraton graph" or the workng curve and was used to nterpolate the concentraton of an "unknown" soluton drectly. In other words the gradent "b" (and the ntercept "a" f applcable) was not even calculated; rather the determnaton of the unknown concentraton was carred out graphcally. It s now qute common to determne the gradent and ntercept usng a computer to carry out the method of least squares. Ths s a mathematcally precse way of drawng a lne of "best ft" through a seres of plotted ponts whch, n general, wll not all fall perfectly on the drawn graph. It s possble to carry out the least squares ft, and hence obtan values for "a" and "b", wthout performng any statstcal analyss. All that results from such a procedure s a value for "a" and "b" wthout any ndcaton as to how "good" these values are. Furthermore unless the graph s drawn as well, there wll be no ndcaton as to how well the data fts a straght lne relatonshp. After all the program n the computer or calculator wll probably carry out the least squares ft to an assumed lnear equaton even f the plotted ponts form a crcle! Now drawng the graph does gve a vsual ndcaton of how well the ponts do ft a straght lne and some sort of measure of ths factor s necessary. It s, of course, possble to carry out some smple statstcal calculatons whch wll gve a quanttatve measure of the "goodness of ft" and when these calculatons are added to the least squares calculaton then the sum total s a regresson analyss. However, before gong on to the statstcal detals there s an mportant and, unfortunately, an often neglected matter to be dscussed. To lead up to ths matter t s frst necessary to gve some statstcal background to the topc of regresson analyss. However, what follows must not be regarded as a rgorous treatment. Consder the equaton y = a + bx. It s assumed the "x" varables are fxed n advance and that any observaton errors are neglgbly small. The varable y does possess a random error, but t s assumed there are no systematc errors. Thus the followng can be wrtten: y = α + ß*x + e () where t s assumed that the true relaton between x and y can be represented by ths equaton. e s the dfference between y and the expected value, Y; the expected value beng that gven by (α + ß*x ). Let Y be the expected value of y so that: e = y -Y () For each value of x the value of Y s taken to represent the average (arthmetc mean) of a theoretcally nfnte number of y values whch are normally dstrbuted. Thus there s a normal dstrbuton of y's for each value of x, the standard devaton of each dstrbuton beng assumed to be the same.
Fnally t s also assumed that e = and the mean of all the e s s also zero. (If the least squares ft lne s accurately calculated then the algebrac sum of all the devatons e from the lne should be zero.) Once the least squares ft lne s calculated then ths s referred to as the lne of regresson of y on x. The other case of a regresson of x on y would nvolve assumng a seres of essentally error free y values and the assocated values of x such that x = α + ß *y + e. Obvously, n practce t s smply a matter of choosng whch varable s to have the seres of fxed values and nomnatng ths as, say, x (the ndependent varable), then always wrtng y = a + ß*x + e where y s referred to as the dependent varable. Note also another name for x s the "explanatory varable" rather than the "ndependent varable". In carryng out the least squares ft we smply wrte y = a + bx where the fnally calculated values of a and b are taken as estmated of the true values α and ß (estmates snce a fnte populaton was taken). Method of Least Squares. The least squares method nvolves the mnmsaton of SSE ( sum of error squares ) where SSE = e = ( y Y ) () and b = n ( x x)( y y) = n = ( x x) =S yx /S xx (4) n n a = y b x = = a nd b are unbased estmators of the true parameters α (5) and ß (Mntab) Let the x data be n column C and the y data n column C mtb> regress C C The number s necessary as ths ndcates the number of predctors, and s always for lnear regresson (see multple regresson secton for more detals). Analyss of varance for Regresson If σ s the varance of regresson (equvalent ot the varablty of the data about the regresson lne) then s yx s an unbased estmator of σ s yx = SSE/(n-) = n = ( y Y ) n (6)
= (S yy -bs xy )/(n-) where S yy = ( y y) n = (7) S yx was defned n equaton 4. Once the least squares ft has been carred out then a number of questons may be asked about the results: () How good are the values of a and b as obtaned by the least squares ft? Obvously the values were determned from a gven sample (of x's and y's) and f the calculaton was repeated wth a dfferent sample then dfferent values of a and b would, n general, be obtaned. Hence t s desrable to have some measure of the error n a and b. () Suppose once a and b are known that a value of y s calculated for a gven value of x. In other words a value for y s beng predcted for some future observaton. The queston that naturally arses-s can two confdence lmts be constructed such that t can be asserted wth a probablty of, say,.95 that these lmts wll contan the next observed value of y? () There s another queston whch s subtly dfferent to () above. The least squares ft lne s, n effect, a seres of calculated y values for each of the values of x chosen. ane mght requre confdence lmts for ths lne.e. two lmts such that one could quote a gven probablty that the least squares lne would be contaned by these lmts. In practce ths means askng how good an estmate s a calculated value of y of the true average value of y for the value of x n queston. Ths queston s related to queston () n that t asks somethng of the calculated lne (whch s defned by a and b n queston ()). Thus the confdence lmts whch show the range wthn whch t s beleved the estmated average y values le ndcate the precson wth whch the average y's have been estmated.e. the varablty of Y (.e the value of y calculated by equaton ). They do NOT ndcate the spread of ndvdual values of observed y s. On the other hand f the nterest s n predctng the next observaton of y (whch, n general, wll not le on the calculated lne) then confdence lmts must be constructed for that specfc event. Ths s what queston () s about and those confdence lmts relate to the varablty, not of Y, but of the error (Y - y ). Ths pont s dscussed further n the case studes. (4) What s the standard devaton of the sample data? Ths queston s a lttle vague as stated. More specfcally, then, one can calculate the devatons of the plotted ponts, Y, from the lne of least squares ft.e. (Y - y ). Then one obtans the standard devaton of these varous dfferences (remember the algebrac mean of the dfferences should be zero f the least squares ft has been accurately carred out). Ths standard devaton, once calculated, obvously gves a measure of how scattered the ponts are, around the lne of least squares ft. Thus ths pont s concerned wth how "good" s the lne of best ft. In fact ths leads to another matter, correlaton,and wll be dealt wth later under that headng. Now the above background has been revsed t s possble to return to the neglected pont mentoned earler. Consder queston () above and suppose the least squares ft was carred out for a graph of absorbance versus concentraton. Obvously once the curve has been constructed the chemst s not n the slghtest bt nterested n predctng the next absorbance value, should he or she prepare another standard calbraton soluton. In fact, the stuaton s qute the reverse. The absorbance of an "unknown" soluton wll be measured and what s requred s a predcton of the concentraton. Usng the symbols y and x as used prevously t s x that must be predcted from y not y from x. Therefore the treatment n standard elementary texts on statstcs s not complete for the ''calbraton problem" snce authors wll commonly, and naturally, assume that a regresson of y on x wll be used to make future predctons about the dependent varable y gven a value for x, not the reverse. So a ffth queston should be added to the above lst vz:
(5) Once the least squares ft has been done then, gven a measurement y, can one form an estmate of how good s the correspondng calculated value of x? How good are the values of a and b? (Queston l) Suppose a and b were determned several tmes usng several sets of sample data. A range of, say, b values would be expected and ths dstrbuton of b values would have some standard devaton σ b. It s possble to form an estmate of ths standard devaton usng the calculatons from a gven least squares ft-(to a gven sample set). If the dstrbuton of b values s consdered to be a normal one then the usual argument of 95% of observed b values lyng wthn +/-σ b of the mean value of b may be used. However, because the estmate of σ b wll usually come from a relatvely small sample t s better to use a t dstrbuton. Once ths s done confdence lmts, say at the 95% level, may be readly calculated for b. Standard Devaton of b Let s b be the estmate of the true standard devaton, σ b,of the gradent of the straght lne, b. s b = s yx x ( x) / n... (8). Confdence lmts for b Suppose the confdence lmts are to be the 95% lmts.e. (l-α )=.95 or α =.5 as the level of sgnfcance. Then the value of t (for a gven number of degrees of freedom) to be used s t for α / snce the confdence nterval wll span a 5% nterval as +.5 ether sde of nomnal value. In general, then, f the confdence level s to be (l-α ), approprate value of t s t for α / (and the requred number of degrees of freedom). The expresson for the confdence lmts s ß = b+/- t s a/ yx ( ( x x) ). 5 = t S s a/ yx xx = t a/ *s b... (9) where S xx = ( x x) where the number of degrees of freedom approprate for t a/ s (n-) Standard devaton of a The standard devaton of a s gven by: ( x) / n s a = s yx ( / n + n x ( x) )... () and α = a +/- a/ yx t s x ns xx = a +/- t a/ *s a... ()
t tests for sgnfcance of α and ß The null hypothess H : ß = can be tested usng a t-test (.e testng for any sgnfcant relatonshp between y and x) The t statstc s calculated as follows: t = b/(s yx /S xx.5 ) (n- degrees of freedom) Note that ths t value s expermentally determned and should be compared wth a value found from t tables.e the crtcal value at α =.5 and n- degrees of freedom. Smlarly, to test whether the regresson lne has a sgnfcantly non-zero ntercept.e H o : α = can be tested usng:- t = a S x ns yx / xx... () Standard devaton of sample data? (Queston (4)) The standard devaton of the sample data as estmated from the "scatter" of ponts around the lne of least squares best ft s s yx whch has been quoted prevously (equaton 6) CORRELATION The above s a small begnnng to the queston of how well the least squares lne actually fts the data. In fact t s qute nadequate because a more detaled consderaton rases the queston as to how much of the total varaton of the y's can be attrbuted to the relatonshp wth x, and how much can be attrbuted to all other factors, ncludng chance. After all one reason the lne may not ft well s that the relatonshp s not lnear. On the other hand to the extent the relatonshp between y and x s lnear then one of the reasons the dfferent y's have dfferent values s smply that y vares lnearly wth x. The rest of the varaton (whch leads to the plotted ponts beng scattered around the least squares lne) may be due to expermental error. It s possble to derve a quantty whch measures the proporton of the total varaton of the y's that can be attrbuted to the lnear relatonshp wth x. Note that the quantty gven below measures the degree of lnear relatonshp. The data may ft a parabola almost perfectly but the lnear correlaton wll be very poor. Coeffcent of Determnaton and Coeffcent of Correlaton The quantty referred to mmedately above s the coeffcent of determnaton. The square root of ths quantty s the coeffcent of correlaton. Symbols used are R and R respectvely, wth R always postve. When the square root s extracted to gve R t s customary to choose the sgn of R to concde wth the sgn of the gradent b. The equaton for R s: r = n xy x y... () ([ n x ( x) ][ n y ( y) ]). 5
= ( x x)( y y) ( x x) ( y y) = S S xx xy S yy = β S xx S yy = SSR/SST where SSR s the sum-of-squares(varance) explaned by the regresson lne and SST s the total sum-of-squares. Thus ( y y) SST = SSR + SSE ( Y y) ( y Y )... (4) The physcal sgnfcance of the calculaton s best gven n terms of R not R. Thus f R =.96 then 96% of the varaton of y can be attrbuted to a lnear relatonshp between y and x. Obvously then, R cannot be greater than, and hence the R cannot exceed. Alternatvely f R = then none of the varaton of y can be attrbuted to a lnear relatonshp wth x. There s no (lnear) correlaton between the x and y values. Now f R does equal then all the plotted ponts wll le on the least squares lne. In ths sense R or R s a measure of how well the ponts le on the lne. A word of warnng, however, f the expermenter assumes there s a lnear relatonshp (on theoretcal grounds, say) then R may be taken as a measure of the scatter of the ponts due to expermental error n the y values. On the other hand f a lnear relatonshp s not assumed then R becomes a measure of the probablty that a lnear relatonshp does exst. R s not, tself, equal to some statstcal probablty, but t can be used to consder the probablty that an apparently hgh value of R obtaned was due to a purely chance varaton of y wth x. For example consder the extreme case of plottng a straght lne wth two ponts only. Snce t s always possble to place a straght lne perfectly through two ponts then R wll come out as unty. However, t scarcely takes any mathematcal statstcs to ndcate that n ths case R = certanly doesn't prove there s a lnear relatonshp between y and x! The matter of testng the value of R or R obtaned, to see at what confdence level one could ndcate that the value had not arsen by chance, wll be consdered later. However, the pont of the "two pont straght lne" - can be generalzed to ndcate that the smaller the number of data the less fath can be placed n an apparently "good" value of R. Put n another way t can be stated that the successve elmnaton of pars of x and y values s always lkely to ncrease the value of R. The moral s that the value of R obtaned must always be consdered n terms of the number of pars of x and y values taken. Another pont s that t requres some experence to judge what s a "good" value of R when a lnear relatonshp s assumed and the value of R s beng used as an ndcaton of the scatter of ponts around the least squares lne. It s very common s plottng workng curves of absorbance versus concentraton to smply take fve or sx standard solutons. Tradtonally a pece of foolscap szed graph paper would be used and a vsual judgement made as to the degree of scatter of the plotted ponts. Experence has shown that graphs whch are judged as beng rather poor by ths subjectve method nevertheless stll gve an R value of at least.98. In such cases the frst two fgures n the R value are of no use n makng any judgements and the thrd and fourth decmal places must be consdered. Undoubtedly the small number of data pars contrbutes to ths stuaton. It should also be remembered that R s not a lnear measure of the degree of correlaton. Thus R =.8 s not twce as "good" as r =.4. Indeed for R =.8 and.4 then R =.64 and R =.6respectvely..e. n the sense of the percentage of the varaton of the y's whch can be attrbuted to the relatonshp wth x, then R =.8 s four tmes as strong a correlaton as.4. An F test can be performed to test the sgnfcance of the regresson also:- F = MS(regresson)/MS(error) where MS(regresson) = SSR/d of f and MS(error) SSE/d of f
but F = t equvalent where t was defned above for the null hypothess H : ß = so both tests are note: Mntab uses an adjusted R as ths s an unbased estmator of the true coeffcent of regresson where adjusted R = SSR(n-p)/SSE(n-) p = no. of coeffcents (= for lnear regresson) and n s the number of data pars Confdence Lmts (Intervals) for the Lne of Best Ft and Predcton Lmts (see Questons () & ()) The confdence lmts relate to the problem of estmatng the true average value of y from a gven value of x. Ths s because the lne of best ft s, statstcally, an estmate of the average values of y correspondng to the x values. The confdence lmts then gve a measure, at a desred probablty level, of how good s the estmate. We are wrtng here of y = a + bx wth y the ndependent varable. In fact there s a dstrbuton (assumed normal n ths dscusson) of y's for each x and so f we attempt to predct a gven value of y correspondng to a chosen x value then the uncertanty of that predcton from the best ft lne s greater than the uncertanty of the true average value of y. In other words snce the best ft lne s an estmate of true average y values there s a further uncertanty as to what any one partcular y value wll be for a chosen value of x. The-equatons for the confdence lmts of the lne (.e. of y average, also called y calculated ) and the predcton lmts for a partcular value of y are as follows: Confdence Lmts about y average The lmts are gven by: x x/ n. 5 +/- tn, α / syx[ + ]... (5) n x ( x) / n these beng +/- about the y average lne.e. the lne of best ft. x n the equaton s the chosen value of x for whch y average s to be estmated. If these lmts are plotted for a range of x values they wll be found to be narrower towards the mddle of the best ft lne. Predcton Lmts for y The expressons for these lmts are agan very smlar to the confdence lmt equatons. All that need be changed s that the term l/n under the square root be replaced by (n+l)/n. Ths s equvalent to addng "" to the entre term under the square root. The lmts then are: x x n 5 +/- tn syx + /., α / [ + ]... (6) n x ( x) / n Of course the addton of the "" makes the term larger and hence the predcton lmts are wder than the confdence lmts. Confdence Lmts for determnng x from y Once the regresson equaton s known t s very smple to calculate an x value correspondng to a measured y. For example a common procedure n analytcal chemstry s to construct a calbraton lne from measurements on a set of standards and to use ths lne to predct the concentraton of an
unknown, after measurng the property used to construct the lne (e.g absorbance). The estmaton of confdence ntervals for such a determnaton s qute complex as t nvolves use of both slope and ntercept, each of whch have errors assocated wth them. However an approxmate formula can be used:- s x = s b yx }. 5 ( y y) { + + n b ( x x)... (7) In ths equaton y s the expermental value of y from whch the concentraton value x o s to be determned and s s the estmated standard devaton of x. Ths equaton actually gves the predcton lmts for x whch are +/-t.s x where t s the crtcal t- value at n- degrees of freedom. If several readngs were averaged to get y then we can get the confdence nterval for x by replacng by /m n the formula for s x where m s the number of determnatons of y. Predcton Lmts or Confdence Lmts? It may well be asked when one should use confdence lmts or predcton lmts. In the case of predctng y from x the decson s smply based on what one wants. A predcton of an average y value (strctly "estmate of a true average y correspondng to that chosen value of x") OR a predcton of a partcular y value.e. any one value from a normally dstrbuted set of y's about the average y. Naturally the uncertanty n the latter s greater than n the former case. It should be remembered at ths stage that n the constructon of the best ft lne the y values are assumed to be average y values. In the reverse case of estmatng x from y the answer to the equaton posed n the sub headng depends on how y s nterpreted. Presumably f y s taken as the mean of many observatons then the correspondng value of x may be taken as an average x value. In that case one has confdence lmts n mnd. However one partcular observaton of y can only lead to predcton lmts for x wth the correspondng ncreased uncertanty n that value. Case Study: Calbraton of a Nephelometer A nephelometer s to be calbrated so that ts scale readngs can be converted to ppm solds. The calbraton solutons are prepared by approprately dlutng a standard stock suspenson of very fne slca partcles wth dstlled water. The stock suspenson has a known ppm solds content as determned by a gravmetrc method. The followng data were obtaned from the standard suspensons: X, ppm solds Y, scale readng..5. 8.4 45.5 6.6 76.7 8
C C C C4 X Y Calc Y resdual.. -..5 9.6.8. 8 7..79 4.4 45 48.94 -.94 5.5 6 6.67. 6.6 76 7.4.6 7.7 8 84. -. Regresson Analyss The regresson equaton s Y =. + 7 X Predctor Coef StDev T P Constant.5.88.89.47 X 7.9 5.67.7. S =.4 R-Sq = 99.% R-Sq(adj) = 98.8% Analyss of Varance Source DF SS MS F P Regresson 58.4 58.4 55.4. Resdual Error 5 49. 9.9 Total 6 59.7 Obs X Y Ft StDev Ft Resdual St Resd....9 -. -.94.5. 9.6.67.8.7. 8. 7..5.79.7 4.4 45. 48.94.9 -.94 -.6 5.5 6. 6.67.4.. 6.6 76. 7.4.65.6.5 7.7 8. 84..4 -. -.89
Calbraton of a Nephelometer Y =.55 + 7.9X R-Sq = 99. % 5 Y Regresson 95% CI 95% PI.....4.5.6.7 X Resdual Frequency 4 - - - -4 Hstogram of Resduals -4 - - Calbraton of a Nephelometer Normal Plot of Resduals Normal Score Resdual 4 Resdual Resdual - 4 - - - -4 I Chart of Resduals 4 5 Observaton Number Resduals vs. Fts 4 5 6 7 8 9 Ft 6 7.SL=.5 X=. -.SL=-.5 From the Mntab output we have: Slope (= b) = 7. Intercept ( = a) =. s.d. of regresson (= s yx ) =.4 R (adj) = 98.8% The regresson equaton s y =.*x + 7. Confdence ntervals for the slope: ß = 7. +/- t.5,5.s b From tables t.5,5 =.57 so ß = 7.+/-.57*5.7 = 7. +/-. Confdence ntervals for the gradent: α =. +/- t.5,5 *s a =. +/-.57*.9 sgnfcant fgures) =. +/- 5.88 (or. +/- 5.9, takng note of Tests of sgnfcance: (a) s there a sgnfcant relatonshp between y and x? (.e. H : ß = ) The t-rato for ths test s.7 (and probablty, p, that H s true s zero) so we reject H and accept there s a sgnfcant relatonshp. Alternatvely the F rato from the ANOVA table s 55. (=.7 ) and p =.
(b) Could the lne be consdered to go through the orgn? (H : α = ) t s.89 wth p =.47 that H s true, >.5 so we cannot reject H.e the ntercept s not sgnfcantly non-zero and we can accept the lne passes through the orgn Confdence and Predcton lmts for y: Usng as an example predctng a value for y for x =.5, the predcted value of y from the regresson equaton s 6.7. The confdence lmts for y are 57. to 64.. Ths represents the confdence nterval for the average value of y at ths x value. The predcton lmts for a sngle value of y s the predcton nterval (5.89-69.45). The graph of the CI s and PI s over a whole range of x s s shown. Note that the uncertantes ncrease at the ends of the range, as expected. Confdence ntervals for x predcted from y. For chemsts, t s far more common to wsh to use the lne to predct x at a measured y (e.g predct the concentraton of analyte n an unknown from the nephelometer readng). For example, from a scale readng of y = 6 then x = (y -.)/7. =.49. 4. s x = 7 7 ( 6 46. 4).{ + / +. 7. *. 69 average value of all the y s 5 }. (.69 = S xx = ( x x) 46.4 = =.9 so x =.49 +/-.57*.9 =.49 +/-.75 If the y value was an average of m readng s then replace n the above calculaton by /m e.g f 5 readng were averaged for y then s x = 4. 7 5 7 ( 6 46. 4).{ / + / +. 7. *. 69 5 }. =.6 and x =.49 +/-.4 Analyss of Resduals Further nformaton on the model can be obtaned by examnng the resduals e where e = y -Y If the model we have chosen for the data (so far we have only consdered the lnear model) s approprate then t s expected that the resduals are:- () ndependent () e (average) = () constant varance (v) normally dstrbuted. The valdty of these assumptons can be tested by plots of (I) e vs x or Y (but not y correlated wth y!) or () a normalsed plot of e xs x as e s are These plots serve two purposes (a) detecton of problems wth the model (b) detecton of outlers. A lnear model mght be rejected because (a) no correlaton.e. ß = or (b) the lnear model s napproprate ( called lack-of-ft ) due to curvature n the data. Note that lack-of-ft s used for ths case only, not for general lack of correlaton. Examples of dfferent types of patterns that can occur n resual plots are shown n the fgure. Normalsed plots of resduals wll show up lack-of-ft or outlers as loss of lnearty n the plot (see case study ). Examples of types of types of resdual plots s shown n the followng dagram.
Lack-of-ft To test for non-lnearty n the data replcates are needed for at least some of the data ponts. The error sum-of-sqaures (SSE) can then be splt nto two components:- (a) error due to lack-of-ft and () pure error where pure error ss n j = ( y y ) u= ju j for n j replcates of y at x = x j SS(lack-of-ft) = SSE - pure error ss ms(lack-of-ft) = SS(lack-of-ft)/d of f
Case Study : Fluorescence Analyss Investgate the lnear range of the followng fluorescence experment (conc n ppm, fluorescence n ntensty unts). Duplcate measurements were performed so a check on lnearty could be carred out. C C conc I..4 7.6 4 8. 5 4 5. 6 4 6. 7 6.8 8 6 4.6 9 8. 8.6.5. Regresson Analyss The regresson equaton s I =.6 +.47 conc Predctor Coef StDev T P Constant.56.68.7. conc.4696.764 9.67. S =.87 R-Sq = 97.5% R-Sq(adj) = 97.% Analyss of Varance Source DF SS MS F P Regresson 685.4 685.4 87.7. Resdual Error 4.5 4.4 Lack of Ft 4 4..6 5.. Pure Error 6.. Total 78.9 Lack of ft test Possble curvature n varable conc (P-Value =.) Possble lack of ft at outer X-values (P-Value =.) Overall lack of ft test s sgnfcant at P =.
Fluorescence Analyss Y =.5595 +.46964X R-Sq = 97.5 % 4 I Regresson 95% CI 95% PI 4 5 conc 6 7 8 9 Resdual Frequency - - - -4 4 - Normal Score Hstogram of Resduals -4 Normal Plot of Resduals - - - - Resdual Fluorescence Analyss Resdual Resdual - - - -4 - - - -4 I Chart of Resduals 5 Observaton Number Resduals vs. Fts Ft 5 5.SL=.7 X=. -.SL=-.7 4 The Mntab output and plot of the data are shown above. Snce replcates have been performed a test for lack-of-ft can be carred out. The Mntab test for lack-of-ft shows evdence of non-lnearty and nspecton of the plot confrms ths. It should be noted here that,despte an R value of.97, the lnear model s not approprate. An alternatve s to carry out quadratc regresson (see next secton) to ft a curve to the data. Ths mproves R to.988. However s ths the only nterpretaton? Inspecton of the plot of the data shows that the ponts are qute lnear, except for the pont at x =. Thus another alternatve would be to reject these ponts and ft a straght lne to the other data. Ths would be acceptable provdng the model was not used to predct responses for x>8.e operatng only n the lnear range. For ether model, predctng responses for x> would be extremely unrelable. Multlnear Regresson So far we have only consdered the case where y s predcted by only one varable, x. There are cases where more than one varable may be used to predct y (very common n expermental desgn studes). Thus we can have a model of the type:- y = ß + ß x +... + ß m x m + e In a model such as ths t s a necessary condton that the x s are uncorrelated (or orthogonal) Matrx approach to Least Squares Regresson
The equatons for calculatng the ß s usng least squares are very complex when there are more than two parameters and can more easly be expressed n matrx notaton. Consder frst the lnear case:- Y = X*B + E y β x where Y =. B = β X =... y n x nx x nx e E =.. e n B s estmated by (X T *X) - *X T *Y provdng (X T *X) - exsts (f any of the x s are correlated then the matrx s sngular and no nverse exsts ). X T s the transform of X (rows -> columns). Ths formula can be generated to multlnear regresson, for m predctors β x x m B =. X =. x n x mn β m for y = ß + ß x +... + ß m x m + e Specal case: the quadratc model can be dealt wth:.e. y = ß + ß x + ß x by settng x = x and x = x
Case Study : Determnaton of Cyande n Waste Water. The followng data s for a spectrophotometrc method for determnng cyande n waste water. C Conc C Abs..49.5 4 5.58 5 7.56 6 9.46 7.56 8.67 9 5.76 7.86 9.956.5.58 4 5.45 Regresson Analyss The regresson equaton s abs =.588 +.5 conc Predctor Coef StDev T P Constant.5878..77. conc.569.98 7.9. S =.685 R-Sq =.% R-Sq(adj) =.% Analyss of Varance Source DF SS MS F P Regresson.85.85 47486.4.
Resdual Error.6. Total.9 Lack of ft test Possble curvature n varable conc (P-Value =.) Overall lack of ft test s sgnfcant at P =. Determnaton of Cyande n Waste Water Y = 5.88E- + 5.E-X R-Sq =. %. abs.5. 5 conc 5 5 Determnaton of Cyande n Waste Water Normal Plot of Resduals I Chart of Resduals...SL=.66 Resdual. Resdual. X=. -. -. -.SL=-.66 - - Normal Score 5 Observaton Number 5 Hstogram of Resduals Resduals vs. Fts 4 Frequency -.5-.-.5..5..5 Resdual Resdual.. -...5 Ft. Examnaton of the Mntab output at frst shows excellent agreement wth a lnear model, wth R = % (ths s actually rounded off from 99.97%). However the test for lack-of-ft s sgnfcant! Examnaton of the resduals plot and the normal plot also confrm that there s curvature n the data (compare the resduals plots wth the examples of plots shown earler). The data can be ftted to a quadratc model as shown n the followng Mntab output: mtb> let C = C*C mtb> regress C C C
Regresson Analyss The regresson equaton s abs = -.59 +.55 conc -. conc^ Predctor Coef StDev T P Constant -.59.44 -.6. conc.5469.474.. conc^ -.98.9-5.9. S =.79 R-Sq =.% R-Sq(adj) =.% Analyss of Varance Source DF SS MS F P Regresson.89.94 77.9. Resdual Error.. Total.9 Source DF Seq SS conc.85 conc^.4 One problem occurs when usng a quadratc model s f a value of x has to be predcted from a value of y. Ths means a quadratc equaton has to be solved. One way around ths s to swtch the data so x and y are reversed (.e do regresson of y on x). Ths wll make predcton smpler but wll not gve a vald predcton of the error snce, n the least squares method, we are assumng the error s assocated wth y and there s no error n x.