ME 40 MECHANICAL ENGINEERING REGRESSION ANALYSIS Regreio problem deal with the relatiohip betwee the frequec ditributio of oe (depedet) variable ad aother (idepedet) variable() which i (are) held fied at each of everal value. The techical term regreio ha become part of the laguage of tatitic due to the work of Sir Fraci Galto. I the 880, Galto laid the foudatio of moder correlatio techique i a tud of the relatiohip betwee the average height of childre ad the height of their paret. I tatitic, regreio mea impl average relatiohip, ad thu betwee two variable, ad, regreio of variable o variable implie a relatiohip betwee the average of the value of the variable for a give value of the variable, ad the value of the variable. Regreio of o Page 67
ME 40 MECHANICAL ENGINEERING Note that the preumptio of beig depedet ad idepedet for repective variable ad, doe ot ecearil mea that a caualit (a caue-effect or a iput-output relatiohip) mut eit betwee them, eve though the ma highl correlated. Becaue, both variable ma be affected b the ame caue, or two variable ma be iterdepedet, or oe variable i the caue, although ot ecearil the ole caue, of the other. Note alo that a regreio of o ca ol be ued to etimate the mea value of for a give, ad hould ot be ued to etimate the mea value for for a give. I other word, a regreio of o doe ot immediatel impl a regreio of o However, there ma eit a regreio of o but i a totall differet ature. I the practice of egieerig eperimetatio, the regreio aali i ued to etimate the bet empirical cotat, with their repective cofidece limit, to fit a mathematical model to a et of meauremet data. Oce a mathematical model i o etablihed betwee the depedet variable ad the idepedet variable(), it ca the be ued to predict for ew value of, b treatig a a cotiuou variable i the implied iterpolatio proce ivolved. Page 68
ME 40 MECHANICAL ENGINEERING Liear Regreio: It i the regreio i which the model ued i liear; i.e., a + b Curviliear Regreio: It i the regreio i which the model ued i a polomial i ; i.e., f() Noliear Regreio: It i the regreio i which the model ued i oliear (polomial or ot); e.g., a + be -c Multivariate (Multiple) Regreio: It i the regreio i which there eit multiple idepedet variable ued i a liear model; e.g., a + a + b or i a oliear model; e.g., a(- ) + b 3 Page 69
ME 40 MECHANICAL ENGINEERING Method of Leat Square: It i a geeral method ued i a ver broad cla of egieerig problem like curve fittig, parameter etimatio, tem idetificatio, ad tatic ad damic optimizatio It major advatage over other techique i that it reult i a et of liear algebraic equatio i term of ukow model parameter if thee parameter appear liearl i the mathematical model. Eample: Let,,.., cotitute meauremet. A bet etimate bet of thee meauremet i aked i the ee that the quatit i miimized. E ( i bet) i The miimizatio of E with repect to bet require that de d bet d ( ) i i d bet bet ( ) 0 i i bet i i i i bet i i i bet bet which i othig but the arithmetic mea of the data et give. 0 Page 70
ME 40 MECHANICAL ENGINEERING Eample (Liear Regreio): Let (, ), (, ),.., (, ) cotitute paired data poit. A bet liear regreio of o a a + b i aked (i.e., the bet etimate of a ad b are aked) i the ee that the quatit i miimized. [ ] E i ( ai + b) i Data Poit i i ( i, i ) a i +b Liear Regreio of o a + b i Note that the quatit i -(a i +b) i the vertical ditace betwee the data poit i ad the regreed value of (a i +b) at i i the veru plae. The miimizatio of E with repect to a ad b require that de da 0 ad de db 0 Page 7
ME 40 MECHANICAL ENGINEERING The olutio of lat two equatio i term of a ad b give: b or b defiig or a a ii i i i i i i i i i i i ii i i i i i i i i i i i i. ad b i i i i i i a b.. Note alo that b - a Page 7
ME 40 MECHANICAL ENGINEERING Eample: Let the followig data how the reult of a eperimet a 6 pair of value. i 3 4 5 6 i 0.5.0.5.0.5 3.0 i 0.7.33.68.88.3.48 3.00.50.00.50.00 0.50 0.00 0.0 0.5.0.5.0.5 3.0 It i deired to obtai the liear regreio of o. The computed value are:.75 ( 0.935).73 ( 0.65) ; 3.79 3.530 to give a 0.685 ad b 0.533 for a liear regreio epreio of o : 0.685 + 0.533 i 0.5.0.5.0.5 3.0 i 0.7.33.68.88.3.48 ti a i +b 0.876.8.56.903.46.588 e i ti - i 0.66-0. -0.9 0.03-0.065 0.08 Page 73
ME 40 MECHANICAL ENGINEERING 3.00.50.00.50.00 0.50 0.00 Regreio Lie 0.685+0.533 0.0 0.5.0.5.0.5 3.0 Correlatio Coefficiet: It i a meaure of the degree of liear correlatio eitig betwee ad. It i defied a: or or r r r ( i )( i ) i ( ) ( )( ) i i i i i i ( ) ( ). or if the regreio coefficiet a ha alread bee calculated r a / i Page 74
ME 40 MECHANICAL ENGINEERING The correlatio coefficiet r alwa lie betwee - ad +. If ad ol if all data poit lie o the regreio lie, the r ±. If r 0, the regreio doe ot eplai athig about the variatio of, ad the regreio lie i horizotal; that i b. The origial defiitio of the correlatio coefficiet ca be iterpreted a iatio due to regreio r var total var iatio where r i referred to a the coefficiet of determiatio. A egative correlatio coefficiet (r<0) i impl a idicatio of a ivere correlatio betwee ad, leadig to a egative lope (a<0) for the regreio lie. For the lat eample, r 0.967 (r0.983), idicatig the eitece of a ver trog correlatio betwee ad. It ca alo be commeted that, the 96.7 % of the total diperio of data poit i from their overall mea value ca be eplaied b the eitig regreio betwee ad. Page 75
ME 40 MECHANICAL ENGINEERING Stadard Deviatio of Data From the Regreio Lie (Stadard Error of Etimate): The deviatio of each data poit from the regreio lie i writte a: i - ti i -(a i +b) The, a a meaure of the vertical catter (diperio) of data, the tadard error of etimate i defied a:,, i ti i ( ) Note that ca be coidered a the etimate of the variace of left ueplaied b the regreio of o. It i defied with - rather tha - i the deomiator becaue two degree of freedom are ued i etimatig a ad b. The followig are two equivalet epreio for, ued i it calculatio if a regreio i alread doe: ( a ),, For the lat eample,, 0.33. ( r ) Page 76
ME 40 MECHANICAL ENGINEERING Stadard Error of Slope: a,,. For the lat eample a 0.064 Stadard Error of -Itercept: b, + ( ),. For the lat eample b 0.4 Stadard Error of Mea Value of t @ a : t,. + ( ) For the lat eample t 0.096 @ 3 Stadard Error of Etimatio of t @ a : t,. + + For the lat eample e 0.64 @ 3 ( ) Cofidece Limit: Aume Gauia ditributio with a degree of cofidece 90 % z.645 Page 77
ME 40 MECHANICAL ENGINEERING Cofidece of Slope: a ± z a For the lat eample: a 0.685 ±.645*0.064 3.00.50.00.50.00 0.50 0.00 Cofidece of Slope a 0.685 ± 0.05 0.0 0.5.0.5.0.5 3.0 Cofidece of -Itercept: b ± z b For the lat eample: b 0.533 ±.645*0.4 3.00.50.00.50.00 0.50 0.00 Cofidece of -Itercept b 0.533 ± 0.04 0.0 0.5.0.5.0.5 3.0 Page 78
ME 40 MECHANICAL ENGINEERING Cofidece of Mea Value of t at (3): t ± z t For the lat eample: t.588 ±.645*0.096 t.588 ± 0.58 Cofidece of Etimatio of t at (3): (i.e., the ize of the bad at (3) which i epected to iclude 90 % of the ew data poit) e ± z e For the lat eample: e.588 ±.645*0.64 e.588 ± 0.70 Page 79