8. SIMPLE LINEAR REGRESSION. Stupid is forever, ignorance can be fixed.

Transcription

1 CIVL 33 Appomto d Ucett J.W. Hule, R.W. Mee 8. IMPLE LINEAR REGREION tupd s foeve, goce c be fed. Do Wood uppose we e gve set of obsevtos (, ) tht we beleve to be elted s f(): Lookg t the plot t ppes tht Y ves lel wth X, but the dvdul obsevtos cot some smll mout of dom eo. If ou supposto s tue, we c model ths dt wth the mthemtcl epesso whee Y ˆ + X + e ˆ ˆ ˆ + ˆ s the equto of stght le (ou egesso model) d e s omll-dstbuted dom eo wth me vlue of zeo d vce σ. The dvdul e e clled esduls. The e the dscepces betwee the ctul dt d ou fuctol ppomto of the dt: ( ) e ŷ + Ou tsk s to fd the oe uque le tht mmzes these esduls fo the ete dt set. The best ppoch s to mmze the sum of the sques of the esduls, whch we ll cll : ( e ) ( ) Recll tht mm occu whee the fst ptl devtves of fucto e equl to zeo. Ou s fucto of the vbles d so we c deteme whee s mmum b settg,

2 Recllg tht the devtve of sum s the sum of the devtves, we eed to fd d such tht: ( ) ( )( ) We c ege these two equtos s follows: d move the costt coeffcets d to the outsde of the summtos to get ce ll the summtos eld costt vlues, ths s just sstem of two smulteous le equtos two ukows ( d ) d c esl be solved b hgh school studet s d Ths s the oe uque le tht fts the obsevtos close th othe le. Of couse, we stll do t kow how well t fts the dt, ol tht t fts the dt bette th would othe le. We ll look t ws to ssess the dequc of egesso model fte emple.

3 Emple A chemcl egee s vestgtg the effect of opetg tempetue o the eld fom some chemcl pocess. A lboto stud poduces the followg obsevtos: Tempetue ( C) Pocess Yeld (%) Afte plottg the dt, ou feel cet tht thee s le eltoshp betwee the pocess tempetue d the pocess eld: Pocess Yeld (%) Tempetue ( o C) The followg quttes c be computed fom ths dt: ,5 48,5,33 Wht e the slope d tecept of the le model?

4 Mesues of Vto Regesso I the lst secto, we foud the equto of le tht fts the obsevtos bette th othe le. To deteme just how good bette th othe ell s, we ll beg b defg sevel mesues of vto tht we ll use to ssess how well ou model fts the obseved dt. The fst mesue of vto s the totl sum of sques T ( ) whch s mesue of the vto of the obseved vlues bout the ow me. Ths totl c be boke to two compoets. The egesso sum of sques R ( ˆ ) s the vto of the egesso pedctos bout the me of the obseved vlues. It c be thought of s mesue of the epled vto the vto the obseved tht s epled b the poposed fuctol eltoshp betwee d. The eo sum of sques E ( ŷ ) s mesue of the vto of the obseved vlues bout the egesso le. It c be thought of s mesue of the uepled vto the vto the obseved tht cot be epled b the poposed fuctol eltoshp betwee d. f() Uepled vto Epled vto - 8 -

5 The cocepts of epled d uepled vto c be used to develop sevel mesues of the goodess of ft of ou egesso model. Let s etu to ou ogl dt obsevtos fo mute. If we dd t kow thee ws eltoshp betwee d d we wee sked to estmte the vlue of, we would hve o choce but to use the thmetc me of the vlues. Gve the ge of the vlues ehbted ou dt, howeve, ths could led to cosdeble eo. We c qutt ths eo b computg the totl sum of sques T ( ) As t tus out, we ve detemed tht the espose vble Y s ctull fucto of some egesso vble X d we ve modeled tht eltoshp s stght le: I dog so, we ve sgfctl educed the mout of uepled sctte of the obsevtos. Obseved vlues of Y tht dffe fom e epled b the coelto wth X. We c qutf the mout of the epled sctte b computg the egesso sum of sques: - 8 -

6 R ( ˆ ) The to of the epled vto the to the totl vto the s clled the coeffcet of detemto R T If the le egesso model fts the dt pefectl, R T d, whch mes tht % of the vblt s epled b the le model. Lkewse, f.95, the 95% of the vblt s epled b the le model. If, o the othe hd, the le egesso model does t povde bette estmte of th dd the thmetc me, R d. We c use the sme mesues of vto to compute othe sttstcs tht elte to the goodess of ft of ou egesso model. B ow ou should hve ecogzed the eltoshp betwee T d the stdd devto of the obseved vlues bout the me: s T ( ) The stdd eo of the estmte s E ( ŷ ) s sot of stdd devto of the obseved vlues bout the egesso le ft to the dt. It s mesue of how tghtl the dt fts the egesso le. If the obseved dt e ot ell lel elted, the stdd eo of the estmte wll ppoch the stdd devto of the obseved dt. If, o the othe hd, the obseved dt e tul lel elted, the stdd eo of the estmte wll ppoch zeo. Note tht T hs ( ) degees of feedom becuse T depeds o oe pmete ( ) tht s computed fom the dt tself. Lkewse, E hs ( ) degees of feedom becuse E depeds o two pmetes ( d ) tht e computed fom the dt tself. If we ol hd two dt pots, T would hve just oe degee of feedom d E would hve o degees of feedom. Wht does t me to hve just oe degee of feedom o zeo degees of feedom? - 8 -

7 Emple Usg the egesso model ou deved fo the chemcl pocess poblem, clculte the stdd devto of the elds, the stdd eo of the estmte, d the coeffcet of detemto. ŷ ( ) ŷ ( ŷ ) ŷ ŷ ( ) Σ 9 Σ 8.83 Σ

8 Thee e sevel ws to tepet these esults. The vlue tells us tht le eltoshp betwee pocess eld d tempetue epls 95% of the vto the obseved elds. The emg 5% s uepled vto tht could be due to mesuemet ccuc o pehps mssg vble o two tht hve ot bee ccouted fo. The s vlue tells us tht the stdd eo of the esduls bout the egesso le s 3.8% eld. Thk of ths s stdd devto. The esduls e omll dstbuted dom eos wth me vlue of zeo d stdd devto of s. We kow fom the fst hlf of the clss tht ppomtel /3 of ll dt should fll wth oe stdd devto of the me f the dt s omll dstbuted. Theefoe, fte we plot the egesso le ˆ.47. we should fd tht ppomtel /3 of the obsevtos should fll wth ±3.8 pecetge pots of the le. The plot below shows tht 5 of the obsevtos e wth ±3.8 pecetge pots of the le whle of the e clel futhe w. Thus, somewhee betwee 5% d 8% of the dt flls wth ±3.8 pecetge pots of the le. plttg the dffeece, we c s tht 65% (mght s well cll t /3) of the dt flls wth ±s of the le. 9 Pocess Yeld (%) Tempetue ( o C) We lso kow tht 95% of the obsevtos should fll wth ±s of the egesso le. ce ou c t splt obsevtos, ths mes tht ethe 9 o obsevtos fll wth ±6.36 pecetge pots of the egesso le. I ths cse, ll obsevtos fll wth the stted boudes

9 EXCEL HINT: REGREION Thee e sevel ws to do egesso Ecel. If ou e og hu, the quckest w s to plot the dt s XY (ctte) plot d use Ecel s Tedle fetue to deteme the best ft stght le. Whe ou select the commd Add Tedle fom the Cht meu, ou e peseted wth dlog bo wth two tbs, Tpe d Optos: Fo ow, just select Le ude the Tpe tb (lte, we ll le ll the othe egesso tpes) d check the boes fo Dspl equto o cht d Dspl R-squed vlue o cht ude the Optos tb. Ecel wll clculte the egesso le d dspl ts equto d the esultg coeffcet of detemto tet bo somewhee wth the plot e of the cht. You c lso use the Regesso tool the Dt Alss Toolpk. elect Dt Alss fom the Tools meu, the select Regesso d ou e peseted wth dlog bo: Fo ow, just tell Ecel whee t c fd the Y dt d the X dt d whee t c put the output tble. The output tble wll be sml to the oe show o the et pge:

10 Obvousl, ths povdes much moe th just the slope d tecept. It povdes pletho of sttstcs tht c be used to judge how good the egesso model s d how much cofdece ou c hve ts pedctos. B the ed of the semeste, we ll hve coveed evethg ths tble. Fo ow, though, we ll cocette o the sttstcs we ve led dscussed. The coeffcet of detemto,, s cell B5 d the stdd eo of the estmte, s, s cell B7. The totl umbe of dt pots,, s cell B8. Cell C cots the egesso sum of sques, R, d cell B cots the umbe of degees of feedom of the egesso sum of sques (whch s oe hee becuse thee s just oe egesso vble). Cell C3 cots the esdul o eo sum of sques, E, d cell B3 cots the umbe of degees of feedom of the eo sum of sques, whch we sd befoe s (whch s eght hee becuse thee e te obsevtos). Cell C4 cots the totl sum of sques, T, d cell B4 cots the umbe of degees of feedom of the totl sum of sques, whch s (whch s e hee). The tecept,, s cell B7 d the slope,, s cell B

11 You c lso use Ecel s Fucto Wzd to clculte sevel petet egesso sttstcs: LOPE(kow_'s,kow_'s) INTERCEPT(kow_'s,kow_'s) CORREL(kow_'s,kow_'s) TEYX(kow_'s,kow_'s) Retus the slope of the le egesso le ft to - dt stoed the kow_'s d kow_'s cell ges. Retus the tecept of the le egesso le ft to - dt stoed the kow_'s d kow_'s cell ges. Retus the coelto coeffcet betwee the dt s stoed the kow_'s d kow_'s cell ges. Ths s just the sque oot of the coeffcet of detemto! Retus the stdd eo of the estmte fo the egesso le ft to the - dt stoed the kow_'s d kow_'s cell ges. Thee e lso host of fuctos tht c help ou clculte egesso sttstcs usg the fomuls gve ths hdout: UM(_) UM ( ) UMQ(_) UMQ ( ) DEVQ(_) ( ) DEVQ UMXMY(_,_) UMXMY ( ) UMXMY(_,_) UMX MY ( ) UMXPY(_,_) UMX PY ( + ) UMPRODUCT(_,_) UMPRODUCT ( ) These mght keep ou fom hvg to cete s m dffeet colums to hold dffeeces d sques d so foth. Just mke sue ou kow how to use ech befoe ou use them to compute homewok swes