Laboratory 3: Method of Least Squares Introducton Consder the graph of expermental data n Fgure 1. In ths experment x s the ndependent varable and y the dependent varable. Clearly they are correlated wth each other and the correlaton seems to be lnear. We would lke to fnd the lne of the form y( x) a bx (1) that best fts ths data, [Whch n truth actually consst of pars of measurements (x, y )]. Whle we can eyeball the lne t would be nce to have an objectve well establshed method to determne the values for a and b. The most common method to do ths s called the method of least squares. Fgure 1: Plot of Y versus X Our task s to determne the coeffcents a and b n such a way that the dscrepancy between the values of our measurements y, and the correspondng ftted values y(x ) are mnmzed. We cannot determne the coeffcents exactly wth only a fnte number of observatons, but we do want to extract from these data the most probable estmates for the coeffcents. ote that ths technque assumes that the uncertantes are all n the y varable and follow Gaussan statstcs as s often the case for uncertantes arsng from fluctuatons n repeated readngs of the nstrumental scale caused by settngs are not exactly reproducble. These uncertantes are called nstrumental regardless of whether they are due to mperfectons n the equpment or to human mprecson.
Method of Maxmum Lkelhood Our data consst of a sample of observatons extracted from a parent populaton whch determnes the probablty of makng a partcular observaton. Let us defne parent coeffcents a o and b o such that the actual lnear relatonshp between y and x s gven by y0 ) o 0 ( x a b x () For any gven value x we can calculate the probablty densty Ω (a, b) for makng the observed measurement y, by assumng a Gaussan dstrbuton about the actual value y o (x ) wth a standard devaton σ = σ,.e. ( y y0 ( x )) 1 P( x ) e (3) The probablty of makng smultaneous measurements of all y s the product of these probabltes 1 1 a0, b0 ( x ) e 1 y y ( 0 x ) (4) Of course, we do not know the parent dstrbuton values for a o and b o, but for any estmated values of the coeffcents a and b, we can calculate the probablty densty for makng the observed set of measurements as y 1 a, b e (5) where Δy = y - (a + bx ). The method of maxmum lkelhood conssts of makng the assumpton that by maxmzng equaton (5) for the observed set of measurements we are most lkely to obtan the best estmates for a o and b o. Maxmzng the probablty Ω(a, b) s equvalent to mnmzng the sum n the exponental. We defne the quantty ch squared, χ, to be the sum n the exponental: y (6) 1
We have used the same symbol χ, defned n Experment, because ths s essentally the same defnton n a dfferent context. Our method for fndng the optmum ft to the data wll be to mnmze ths sum of squared devatons and, hence, to fnd the ft whch produces the smallest χ. ote: Our development assumes the error assocated wth any sngle measurement s the same for all measurements. Modfcatons to the development must be made when ths s not so. Mnmzng χ In order to fnd the values of the coeffcents a and b whch yeld the mnmum value for χ we use the methods of the calculus,.e. χ a = 0 (7) And χ b = 0 (8) Rearranged these equatons yelds a par of smultaneous equatons to be solved for the coeffcents a and b. Ths wll gve us the values of the coeffcents for whch χ s mnmzed. Ths s done wth the determnants below. In these equatons be sure to dstngush the dfference between the square of the sum of x, ( x ), and the sum of the squares of x, x : a = 1 Δ ( x y x x y ) b = 1 Δ ( x y x y ) Δ = x ( x ) (9) Estmaton of Errors In order to fnd the uncertanty n the estmaton of the coeffcents a and b n our fttng procedure, we refer to our dscusson of the propagaton of errors n Laboratory 1. Each of our data ponts y have been used n the determnaton of the parameters, and each has contrbuted some fracton of ts own uncertanty to the uncertanty n our fnal determnaton. Ignorng systematc errors whch would
ntroduce correlatons between the uncertantes, the standard devaton σ z of the determnaton of any parameter z s gven by z z z (10) 1 y 1 y If we assume that the uncertantes are nstrumental and all the same, they can be estmated from the data. Our defnton n Laboratory 1 of the sample varance s, whch approxmates σ, s the sum of the squares of devatons of the data ponts from the calculated mean dvded by the number of degrees of freedom. In ths case, the number of degrees of freedom s the number of data ponts mnus the number of parameters (two) whch we determned before calculatng s. Thus, our estmated parent standard devaton σ = σ s 1 s y a bx (11) 1 ote that t s ths common uncertanty, σ, whch we have mnmzed by our least-squares fttng procedure. The dervatves n equaton (10) can be evaluated by takng the dervatves of equatons (9), and we can fnd an expresson for the uncertantes n parameters a and b,.e. a b x (1) It may not be obvous from these forms, but the larger the number of data ponts the smaller s the error n the quanttes a and b. Usng R to graph and do least squares regresson. The two most mportant commands are xyplot for graphng and lm for fttng a lnear model. At ther most basc they have the syntax: xyplot(y~x, data=data_frame) and lm(y~x, data=data_frame). An example s useful, we wll use the rubberband dataset that comes as part of R. Frst we must brng fastr and rubberband nto our current sesson: requre(fastr) data(rubberband)
ext we wll look at the structure of the dataset: head(rubberband) Graphng From ths you wll see that there are two varables: Stretch and Dstance. Stretch s the one we have control over so t s the ndependent varable and Dstance s the dependent. Use xyplot to graph the data, use the varable names separated by a tlde and specfy the data as comng from the data_frame rubberband. xyplot(.) We wll now complcate the xyplot command by adjustng the type. By default we have been usng type p (for pont). We can make ths explct: xyplot(y~x, data=data_frame, type=c( p )) Usng ths command should have left your graph unchanged. There are other types as well, the other one that s relevant s type r for regresson. xyplot(y~x, data=data_frame, type=c( r )) What s most useful s to make both on the same graph: xyplot(y~x, data=data_frame, type=c( p, r )) Fttng It s useful to see the least-squares regresson lne graphed wth the data to make decsons about how meanngful the ft s, but t s useful to know the coeffcents n the equaton for the lne Try usng lm: Dstance = a + b Stretch. (13) lm(dstance~stretch, data=rubberband) Ths gves the most basc nformaton, the ntercept (a) and the coeffcent that multples Stretch (b). We can extract more nformaton f we look n more detal at the lnear model structure, frst by storng the model and then by lookng at ts summary: flght.model<-lm(dstance~stretch, data=rubberband) summary(flght.model) Ths gves a varety of nformaton ncludng estmates for the coeffcents, uncertantes n the coeffcents, probabltes that the coeffcents values could be explaned by chance nstead of a real correlaton.
The last thng we need s an estmate of the varance n the parent dstrbuton, s. Ths can be estmated from the dfference between the lnear model and the data ponts, namely the resduals. The resdual of each data pont s held by the lnear model varable resduals flght.model$resduals and also returned by the functon resd(lnearmodel) resd(flght.model) From Equaton (11) we want the sum of the resduals squared dvded by the number of degrees of freedom, whch s also a model varable. Castng Eq. (11) n the syntax of R s<-sqrt(sum(resd(flght.model)^)/flght.model$df.resdual) If you look at the summary of our model agan you wll see that ths quantty was already reported as the resdual standard error. Whle you are lookng at that part of the summary you wll also fnd the R value. It s worth notng that ths gves the percent of the varance n the data that s explaned by the model. Expermental Procedure for the General Case Gven the data shown n Table 1 below: Table 1: Expermental data for temperature versus poston along a rod Tral X (cm) T ( o C) 1 1.0 15.6.0 17.5 3 3.0 36.6 4 4.0 43.8 5 5.0 58. 6 6.0 61.6 7 7.0 64. 8 8.0 70.4 9 9.0 98.8 1. Determne the best parameters a and b to the equaton T = a + bx. ote that we are assumng that all of the error s assocated wth a measurement of temperature, not length.
. Determne the standard devaton of the temperature data. 3. Determne the errors assocated wth a and b. 4. Express the thermal gradent n a manner sutable for reportng;.e., the slope of the lne. 5. Plot the data and ft. 6. Use the standard devaton s to draw an error bar on each data pont of temperature ± s. An } ) error bar about a data pont reflects the probablty that another measurement would reproduce the frst value wthn one standard devaton 66% of the tme. To do ths n R, you can use the followng xyplot() command, adjustng X, Y, s, and data to ft your stuaton. xyplot(y~x, data, lb=data$y-s,ub=data$y+s, panel=functon(x, y, lb, ub, ){ panel.xyplot(x, y, type=c( b, r ), ) panel.segments(x0=x, x1=x, y0=lb, y1=ub, ) 7. These data were made up, but a normal thermometer mght have a ΔT = 0.5 o C. How does ths compare to s? Specfc Case for Constrant a = 0 An mportant subset of the general problem dscussed above s the lnear functon constraned to pass through the orgn,.e. y(x) = c x (14) Ths s a lnear problem but t represents a functon whch has a y ntercept of zero. We follow the same approach as above startng wth the defnton of χ, and we proceed as n the prevous secton and fnd the mnmum of χ wth respect to c by lettng: c 0 (15) The standard devaton assocated wth a data pont y n ths case s 1 s y cx (16) 1 1
where the - 1 factor appears nstead of - snce we have only used the data once to determne c. To fnd the error n the slope we fnd the change n c wth respect to y,.e. c y, then we obtan σ c = σ x =1 (17) Thus, equatons (15), (16), and (17) provde us wth the lnear least squares ft of data constraned to go through the orgn. Expermental Procedure for the Specfc Case of Data Passng Through the Orgn Part One: You are gven a set of alumnum dsks wth dfferent dameters. 1. Measure and record the dameter of each dsk and label these data ponts d.. Devse a technque for measurng the crcumference of the dsk and label these data ponts as crc. 3. Determne the best parameter c ft to the equaton crc = c dam. In R ths s done by gvng the ntercept an explct value of 0 Dsk.model<-lm(crc~dam+0, data=dsk) ote that we are choosng to treat the dameter as the ndependent varable, ths s arbtrary and s equally vald the other way around. 4. Determne the standard devaton assocated wth each C. 5. Determne the error assocated wth c. 6. Use your model to predct a crcumference based on the model for a dameter of 10cm. p<-predct(dsk.model, newdata=data.frame(dam=10)) 7. Use your model to predct a crcumference for all dameters n your dataset. 8. Use the results of 7 to plot the data and the ntercept=0 ft, xyplot(crc+p~dam,data=dsk, type= b ) 9. Record the best ft parameter for the slope as well as for the estmated error n the slope. Compare to the known value of and dscuss the sgnfcance of your results for the goodness of your data.
Part Two: You are gven a voltage supply, a decade resstor box, an analog current meter, and a dgtal volt meter. Set the analog current meter to the 50 mllamp scale and the decade resstor box to 10 ohms. Use the dgtal meter to record the resstance of the decade resstor box at ths settng when t s not yet connected to anythng else. ow, connect the voltage supply, the resstor box, and the current meter nto a smple seres crcut. Place the dgtal volt meter n parallel across the termnals of the voltage supply and set t to measure DC voltages up to 00 mv. Usng ths set up: 1. Measure and record the current through the resstor as well as the voltage of the power supply for at least 10 dfferent voltages between 0 and 00 mv. (OTE: be sure not to exceed 00 mv so as not to damage the current meter).. Plot the voltage versus current data you collected usng R and fnd what t determnes to be the best ft parameter for the slope as well as the estmated uncertanly n the slope. Make sure to constran the ntercept to zero when fttng the lne to your data. 3. Usng Ohm s Law (V=IR), compare what you found from the graph for the resstance of the decade resstor box to what the dgtal meter found t to be. If they do not agree wthn the level of your expermental precson, dscuss possble reasons for the dfferences. Where possble ste specfc measurements or estmated values taken from the laboratory that support your suppostons for the sources of the error (.e. do not just say human error n readng the meters and leave t at that.) WHE YOU FIISH: LEAVE THIGS AS YOU FOUD THEM!