Any of 27 linear and nonlinear models may be fit. The output parallels that of the Simple Regression procedure.

Transcription

1 STATGRAPHICS Rev. 9/13/213 Calibration Models Summary... 1 Data Input... 3 Analysis Summary... 5 Analysis Options... 7 Plot of Fitted Model... 9 Predicted Values... 1 Confidence Intervals Observed versus Predicted Residual Plots Comparison of Alternative Models Unusual Residuals Influential Points Save Results Calculations Summary The Calibration Models procedure is designed to construct a statistical model describing the relationship between 2 variables, X and Y, where the intent of the model-building is to construct an equation that can be used to predict X given Y. In a typical application, X represents the true value of some important quantity, while Y is the measured value. Initially, a set of samples with known X values are used to calibrate the model. Later, when samples with unknown X values are measured, the fitted model is used to make an inverse prediction of X from the measured values Y. Any of 27 linear and nonlinear models may be fit. The output parallels that of the Simple Regression procedure. Sample StatFolio: calibration.sgp 213 by StatPoint Technologies, Inc. Calibration Models - 1

2 Sample Data: STATGRAPHICS Rev. 9/13/213 The file galactose.sgd contains data on an experiment performed using a new method for measuring the concentration of galactose in blood. The data is similar to that reported by Neter et al (1998). n = 12 samples with known galactose concentrations X ranging between 1. and 1. were measured. The data are shown below: Known Measured An additional sample of unknown concentration was measured, yielding Y = An estimate of the actual concentration of the additional sample is desired, with a 95% confidence interval. 213 by StatPoint Technologies, Inc. Calibration Models - 2

3 STATGRAPHICS Rev. 9/13/213 Data Input The data input dialog box can be used in 2 ways: 1. Given measurements of samples with known values of X, it can be used to fit the calibration model. The coefficients of the model may be saved for later use. 2. If new measurements are made, the stored coefficients can be used to predict the true value of X. Fitting the Calibration Model Y (measured): numeric column containing the n measured values of the quantity to be predicted. X (actual): numeric column containing the n known values of that quantity. Fitted Model Statistics: left blank when fitting a new model. Weights: optional numeric column containing weights to be applied to the residuals if performing a weighted least squares fit. If the variability of Y changes as a function of X, these weights can be used to compensate for the different levels of variability. Select: subset selection. 213 by StatPoint Technologies, Inc. Calibration Models - 3

4 STATGRAPHICS Rev. 9/13/213 Action: select Fit New Model to estimate a new model from Y and X. Using a Stored Model Y (measured): numeric column (or single number) containing the measured values of the quantity to be predicted. Fitted Model Statistics: column containing the statistics saved from the original model estimation. This would normally have been created using the Save Results option when the model was calibrated. The column consists of the estimated intercept, slope, and other relevant information. Action: select Predict X from Y. 213 by StatPoint Technologies, Inc. Calibration Models - 4

5 STATGRAPHICS Rev. 9/13/213 Analysis Summary When fitting a new calibration model, the Analysis Summary shows information about the fitted model. Calibration Models - measured vs. known Y (measured): measured X (actual): known Number of observations: 12 Linear model: Y = a + b*x Least Squares Standard T Parameter Estimate Error Statistic P-Value Intercept Slope Analysis of Variance Source Sum of Squares Df Mean Square F-Ratio P-Value Model Residual Lack-of-Fit Pure Error Total (Corr.) Correlation Coefficient = R-Squared = percent R-Squared (adjusted for d.f.) = percent Standard Error of Est. = Mean absolute error = Durbin-Watson statistic = (P=.942) Lag 1 residual autocorrelation = Residual Analysis Estimation n 12 MSE MAE MAPE ME E-16 MPE Validation Included in the output are: Variables and model: identification of the input variables and the model that was fit. By default, a linear model of the form Y = a + b X (1) is fit, although a different model may be selected using Analysis Options. Coefficients: the estimated coefficients, standard errors, t-statistics, and P values. The estimates of the model coefficients can be used to write the fitted equation, which in the example is measured = known (2) The t-statistic tests the null hypothesis that the corresponding model parameter equals, versus the alternative hypothesis that it does not equal. Small P-Values (less than.5 if 213 by StatPoint Technologies, Inc. Calibration Models - 5

6 STATGRAPHICS Rev. 9/13/213 operating at the 5% significance level) indicate that a model coefficient is significantly different from. In the sample data, the slope is significantly different from but the intercept is not. Analysis of Variance: decomposition of the variability of the dependent variable Y into a model sum of squares and a residual or error sum of squares. The residual sum of squares is further partitioned into a lack-of-fit component and a pure error component. Of particular interest are the F-tests and the associated P-values. The F-test on the Model line tests the statistical significance of the fitted model. A small P-Value (less than.5 if operating at the 5% significance level) indicates that a significant relationship of the form specified exists between Y and X. In the sample data, the model is highly significant. The F-test on the Lack-of-fit line tests the adequacy of the selected linear model in describing the observed relationship between Y and X. A small P-Value indicates that the selected model does not adequately describe the relationship. In such cases, a nonlinear model could be selected using Analysis Options. For the sample data, the large P-Value indicates that the linear model is adequate. Note: the lack-of-fit test is available only when more than one measurement has been obtained at the same value of X. Statistics: summary statistics for the fitted model, including: Correlation coefficient - measures the strength of the linear relationship between Y and X on a scale ranging from -1 (perfect negative linear correlation) to +1 (perfect positive linear correlation). In the sample data, the correlation is very strong. R-squared - represents the percentage of the variability in Y which has been explained by the fitted regression model, ranging from % to 1%. For the sample data, the regression has accounted for about 99.9% of the variability amongst the measurements. Adjusted R-Squared the R-squared statistic, adjusted for the number of coefficients in the model. This value is often used to compare models with different numbers of coefficients. Standard Error of Est. the estimated standard deviation of the residuals (the deviations around the model). This value is used to create prediction limits for new observations. Mean Absolute Error the average absolute value of the residuals. Durbin-Watson Statistic a measure of serial correlation in the residuals. If the residuals vary randomly, this value should be close to 2. A small P-value indicates a non-random pattern in the residuals. For data recorded over time, a small P-value could indicate that some trend over time has not been accounted for. Lag 1 Residual Autocorrelation the estimated correlation between consecutive residuals, on a scale of 1 to 1. Values far from indicate that significant structure remains unaccounted for by the model. Residual Analysis if a subset of the rows in the datasheet have been excluded from the analysis using the Select field on the data input dialog box, the fitted model is used to make predictions of the Y values for those rows. This table shows statistics on the prediction errors, defined by 213 by StatPoint Technologies, Inc. Calibration Models - 6

7 STATGRAPHICS Rev. 9/13/213 e i y yˆ (3) i i Included are the mean squared error (MSE), the mean absolute error (MAE), the mean absolute percentage error (MAPE), the mean error (ME), and the mean percentage error (MPE). This validation statistics can be compared to the statistics for the fitted model to determine how well that model predicts observations outside of the data used to fit it. Analysis Options Type of Model: the model to be estimated. All of the models displayed can be linearized by transforming either X, Y, or both. When fitting a nonlinear model, STATGRAPHICS first transforms the data, then fits the model, and then inverts the transformation to display the results. Include Constant: whether to include a constant term or intercept in the model. If the constant is removed, the fitted model will pass through the origin at (X,Y) = (,). The available models are shown in the following table: 213 by StatPoint Technologies, Inc. Calibration Models - 7

8 STATGRAPHICS Rev. 9/13/213 Model Equation Transformation on Y Transformation on X Linear y 1 x none none y 2 x square root none Square root-y 1 Exponential 1 x y e log none Reciprocal-Y y x 1 reciprocal none Squared-Y 1 y 1 x square none Square root-x y 1 x none square root Double square root y 2 1 x square root square root Log-Y square root-x 1 x y e log square root Reciprocal-Y square y x reciprocal square root 1 1 root-x Squared-Y square root- square square root y 1 x X Logarithmic-X y 1 ln( x) none log Square root-y log-x y ln(x 2 square root log 1 ) Multiplicative y x 1 log log Reciprocal-Y log-x 1 reciprocal log y ln( ) 1 x Squared-Y log-x y ln( ) square log Reciprocal-X 1 x y / x none reciprocal 1 2 Square root-y y 1 / x square root reciprocal reciprocal- X S-curve 1/ x y e log reciprocal Double reciprocal y x 1 reciprocal reciprocal Squared-Y reciprocal-x / y / x square reciprocal 1 Squared-X 2 y none square 1x 2 Square root-y squared- 2 y 1x square root square X 2 Log-Y squared-x 1x y e log square Reciprocal-Y squared-x 2 y x 1 reciprocal square Double squared 2 y 1 1x Logistic 1x e y 1x 1 e square square y/(1-y) none Log probit y ( 1 ln( x)) 1 ( y ) log (inv. normal) 213 by StatPoint Technologies, Inc. Calibration Models - 8

9 measured STATGRAPHICS Rev. 9/13/213 To determine which model to fit to the data, the output in the Comparison of Alternative Models pane described below can be helpful, since it fits all of the models and lists them in decreasing order of R-squared. Plot of Fitted Model This pane shows the fitted model or models, together with confidence limits and prediction limits if desired. 12 Plot of Fitted Model measured = *known (6.2535, ) known The plot includes: The line of best fit or prediction equation: yˆ aˆ bx ˆ (4) This is the equation that would be used to predict values of the dependent variable Y given values of the independent variable X, or vice versa. Confidence intervals for the mean response at X. These are the inner bounds in the above plot and describe how well the location of the line has been estimated given the available data sample. As the size of the sample n increases, these bounds will become tighter. You should also note that the width of the bounds varies as a function of X, with the line estimated most precisely near the average value x. Prediction limits for new observations. These are the outer bounds in the above plot and describe how precisely one could predict where a single new observation would lie. Regardless of the size of the sample, new observations will vary around the true line with a standard deviation equal to. Prediction of a single value. Using Pane Options, a single prediction can be made and plotted. For example, the above plot predicts the value of X given a sample with measured value Y = The predicted value of X equals 6.516, with 95% confidence limits extending from 6.25 to by StatPoint Technologies, Inc. Calibration Models - 9

10 Pane Options STATGRAPHICS Rev. 9/13/213 Include: the limits to include on the plot. Confidence Level: the confidence percentage for the limits. Predict: whether to predict Y or X. Enter the value of the other variable in the At field. Mean Size or Weight: if the measured value is the average of more than one sample, enter the number of samples m used to calculate the average. Predicted Values The model can be used to predict X given Y or Y given X. In the first case, the output is shown below: Predicted Values for X 95.% Predicted Prediction Limits Y-bar X Lower Upper Included in the table are: Y - the measured value at which the prediction is to be made. Predicted X - the predicted value of X using the fitted model. Prediction limits - prediction limits for X at the selected level of confidence. These are the same values displayed on the plot of the fitted model. 213 by StatPoint Technologies, Inc. Calibration Models - 1

11 Pane Options STATGRAPHICS Rev. 9/13/213 Predict: whether to predict Y or X. Confidence Level: the confidence percentage for the limits. Mean Size or Weight: if the measured value is the average of more than one sample, enter the number of samples m used to calculate the average. Predict At: up to 1 values at which to make predictions. Confidence Intervals The Confidence Intervals pane shows the potential estimation error associated with each coefficient in the model. 95.% confidence intervals for coefficient estimates Standard Parameter Estimate Error Lower Limit Upper Limit CONSTANT SLOPE Pane Options 213 by StatPoint Technologies, Inc. Calibration Models - 11

12 STATGRAPHICS Rev. 9/13/213 Type of Interval: either a two-sided confidence interval or a one-sided confidence bound may be created. Confidence Level: percentage level for the interval or bound. Hypothesis Tests The Hypothesis Tests pane can be used to test hypotheses about the model coefficients. In each case, a t-test is performed. The default tests are shown below: Hypothesis Tests Null hypothesis: intercept =. Alternative hypothesis: intercept not equal. Computed t statistic = P-value = Do not reject the null hypothesis for alpha =.5. Null hypothesis: slope = 1. Alternative hypothesis: slope not equal 1. Computed t statistic = P-value = Do not reject the null hypothesis for alpha =.5. The first test concerns whether or not the intercept equals. If so, the model goes through the origin. A small P-Value (less than.5 if operating at the 5% significance level) would indicate that the intercept was not equal to. In this case, the result is not significant, so the line may well go through the origin. If the slope of the line equals 1, a non-zero intercept would be related to bias in the measurements. The second test concerns whether or not the slope equals 1. For a linear model, a slope of 1 indicates that when the known value changes, the measured value changes by the same amount. A small P-Value would indicate that the slope was significantly different than 1. In the current case, neither null hypothesis is rejected, indicating that a possible equation for the calibration curve is measured = known. Pane Options Intercept: the value of the intercept specified by the null hypothesis. Slope: the value of the slope specified by the null hypothesis. 213 by StatPoint Technologies, Inc. Calibration Models - 12

13 observed STATGRAPHICS Rev. 9/13/213 Alternative: the type of alternative hypothesis. If Not Equal is selected, a two-sided P-value is calculated. Otherwise, a one-sided P-value is calculated. Alpha: the probability of a Type I error (rejecting the null hypothesis when it is true). This does not affect the P-value, only the conclusion stated beneath it. Observed versus Predicted The Observed versus Predicted plot shows the observed values of Y on the vertical axis and the predicted values Yˆ on the horizontal axis Plot of measured predicted If the model fits well, the points should be randomly scattered around the diagonal line. It is sometimes possible to see curvature in this plot, which would indicate the need for a curvilinear model rather than a linear model. Any change in variability from low values of Y to high values of Y might also indicate the need to transform the dependent variable before fitting a model to the data. Residual Plots As with all statistical models, it is good practice to examine the residuals. In a regression, the residuals are defined by e i y yˆ (5) i i i.e., the residuals are the differences between the observed data values and the fitted model. The Calibration Models procedure various type of residual plots, depending on Pane Options. 213 by StatPoint Technologies, Inc. Calibration Models - 13

14 percentage Studentized residual STATGRAPHICS Rev. 9/13/213 Scatterplot versus X This plot is helpful in visualizing any need for a curvilinear model. 2.8 Residual Plot known Normal Probability Plot This plot can be used to determine whether or not the deviations around the line follow a normal distribution, which is the assumption used to form the prediction intervals. Normal Probability Plot for measured Studentized residual If the deviations follow a normal distribution, they should fall approximately along a straight line. 213 by StatPoint Technologies, Inc. Calibration Models - 14

15 autocorrelation STATGRAPHICS Rev. 9/13/213 Residual Autocorrelations This plot calculates the autocorrelation between residuals as a function of the number of rows between them in the datasheet. 1 Residual Autocorrelations for measured lag It is only relevant if the data have been collected sequentially. Any bars extending beyond the probability limits would indicate significant dependence between residuals separated by the indicated lag, which would violate the assumption of independence made when fitting the regression model. Pane Options Plot: the type of residuals to plot: 1. Residuals the residuals from the least squares fit. 2. Studentized residuals the difference between the observed values y i and the predicted values ŷ i when the model is fit using all observations except the i-th, divided by the estimated standard error. These residuals are sometimes called externally deleted 213 by StatPoint Technologies, Inc. Calibration Models - 15

16 STATGRAPHICS Rev. 9/13/213 residuals, since they measure how far each value is from the fitted model when that model is fit using all of the data except the point being considered. This is important, since a large outlier might otherwise affect the model so much that it would not appear to be unusually far away from the line. Type: the type of plot to be created. A Scatterplot is used to test for curvature. A Normal Probability Plot is used to determine whether the model residuals come from a normal distribution. An Autocorrelation Function is used to test for dependence between consecutive residuals. Plot Versus: for a Scatterplot, the quantity to plot on the horizontal axis. Number of Lags: for an Autocorrelation Function, the maximum number of lags. For small data sets, the number of lags plotted may be less than this value. Confidence Level: for an Autocorrelation Function, the level used to create the probability limits. Comparison of Alternative Models The Comparison of Alternative Models pane shows the R-squared values obtained when fitting each of the 27 available models: Comparison of Alternative Models Model Correlation R-Squared Linear % Double square root % Double squared % Double reciprocal % Square root-y logarithmic-x % Multiplicative % Square root-x % Square root-y % Logarithmic-Y square root-x % S-curve model % Squared-Y % Squared-X % Logarithmic-X % Exponential % Squared-Y square root-x % Square root-y squared-x % Reciprocal-X % Squared-Y logarithmic-x % Logarithmic-Y squared-x % Squared-Y reciprocal-x % Reciprocal-Y squared-x % Reciprocal-Y <no fit> Reciprocal-Y square root-x <no fit> Reciprocal-Y logarithmic-x <no fit> Square root-y reciprocal-x <no fit> Logistic <no fit> Log probit <no fit> The models are listed in decreasing order of R-squared. When selecting an alternative model, consideration should be given to those models near the top of the list. However, since the R- 213 by StatPoint Technologies, Inc. Calibration Models - 16

17 STATGRAPHICS Rev. 9/13/213 Squared statistics are calculated after transforming X and/or Y, the model with the highest R- squared may not be the best. You should always plot the fitted model to see whether it does a good job for your data. Unusual Residuals Once the model has been fit, it is useful to study the residuals to determine whether any outliers exist that should be removed from the data. The Unusual Residuals pane lists all observations that have Studentized residuals of 2. or greater in absolute value. Unusual Residuals Predicted Studentized Row X Y Y Residual Residual Studentized residuals greater than 3 in absolute value correspond to points more than 3 standard deviations from the fitted model, which is an extremely rare event for a normal distribution. Note: Points can be removed from the fit while examining the Plot of the Fitted Model by clicking on a point and then pressing the Exclude/Include button on the analysis toolbar. Excluded points are marked with an X. Influential Points In fitting a regression model, all observations do not have an equal influence on the parameter estimates in the fitted model. In a simple regression, points located at very low or very high values of X have greater influence than those located nearer to the mean of X. The Influential Points pane displays any observations that have high influence on the fitted model: Influential Points Predicted Studentized Row X Y Y Residual Leverage Average leverage of single data point = The table shows every point with leverage equal to 3 or more times that of an average data point, where the leverage of an observation is a measure of its influence on the estimated model coefficients. In general, values with leverage exceeding 5 times that of an average data value should be examined closely, since they have unusually large impact on the fitted model. Save Results The following results may be saved to the datasheet: 1. Model Statistics a column of numeric values with information about the fitted model. This column can be used later to predict values of X by selecting Predict X from Y on the data input dialog box. 2. Predicted Values the predicted value of Y corresponding to each of the n observations. 3. Lower Limits for Predictions the lower prediction limits for each predicted value. 4. Upper Limits for Predictions the upper prediction limits for each predicted value. 5. Lower Limits for Forecast Means the lower confidence limits for the mean value of Y at each of the n values of X. 213 by StatPoint Technologies, Inc. Calibration Models - 17

18 STATGRAPHICS Rev. 9/13/ Upper Limits for Forecast Means the upper confidence limits for the mean value of Y at each of the n values of X. 7. Residuals the n residuals. 8. Studentized Residuals the n Studentized residuals. 9. Leverages the leverage values corresponding to the n values of X. 1. Coefficients the estimated model coefficients. Calculations Inverse Predictions xˆ new y ˆ new o (6) ˆ 1 Lower and upper limits for x new are found using Fieller s approach, which solves for the values of xˆ at which the prediction limits new xˆ new x yˆ t / 2, n2 MSE (7) m n S XX are equal to y new, where m is the mean size or weight and xx n i1 2 x x S (8) i Additional calculations may be found in the Simple Regression documentation. 213 by StatPoint Technologies, Inc. Calibration Models - 18