MassLynx 4.1 Peak Integration and Quantitation Algorithm Guide

Copyright notice 2005 WATERS CORPORATION. PRINTED IN THE UNITED STATES OF AMERICA AND IRELAND. ALL RIGHTS RESERVED. THIS DOCUMENT OR PARTS THEREOF MAY NOT BE REPRODUCED IN ANY FORM WITHOUT THE WRITTEN PERMISSION OF THE PUBLISHER. The information in this document is subject to change without notice and should not be construed as a commitment by Waters Corporation. Waters Corporation assumes no responsibility for any errors that may appear in this document. This document is believed to be complete and accurate at the time of publication. In no event shall Waters Corporation be liable for incidental or consequential damages in connection with, or arising from, its use. Waters Corporation 34 Maple Street Milford, MA 01757 USA Trademarks Micromass and Waters are registered trademarks of Waters Corporation. MassLynx is a trademark of Waters Corporation. Other trademarks or registered trademarks are the sole property of their respective owners. Customer comments Please contact us if you have questions, suggestions for improvements, or find errors in this document. Your comments will help us improve the quality, accuracy, and organization of our documentation. You can reach us at tech_comm@waters.com.

Table of Contents 1 Peak Response and Peak Ratio... 1-1 Absolute Response... 1-2 Compound response primary... 1-2 Compound response secondary... 1-2 Compound response both... 1-2 External Response... 1-2 Internal Response... 1-3 Primary/Secondary Peak Ratio... 1-3 2 Calibration Curve Calculations... 2-1 Weighted Calibration Curves... 2-2 Include Origin... 2-2 Average RF... 2-2 Linear... 2-3 Quadratic and Higher Order Curves... 2-4 3 Peak Amount Calculations... 3-1 User Specified Response Factor... 3-2 Average RF Calibration Curve... 3-2 Linear Calibration Curve... 3-2 Quadratic and Higher Order Calibration Curves... 3-3 User Parameters... 3-3 4 Calibration Curve Statistics... 4-1 Coefficient of Determination... 4-2 Curve Correlation Coefficient... 4-2 Table of Contents iii

RRF mean, SD, and %RSD for Average RF curves... 4-3 5 Totals Compounds... 5-1 Peak Response... 5-2 External Response... 5-2 Internal Response... 5-2 Calibration Curve Calculations... 5-3 Peak Amount Calculations... 5-3 6 Limits of Detection (LOD) and Limits of Quantitation (LOQ)... 6-1 Chromatogram Noise Calculation... 6-2 Manual noise measurement... 6-2 Automatic noise measurement... 6-2 Chromatogram Area Noise... 6-3 Response Value for Noise... 6-3 LOD and LOQ Concentrations... 6-4 LOD and LOQ Flags... 6-4 T-LOD and T-LOQ Concentrations... 6-4 7 Peak Statistics... 7-1 Peak area... 7-2 Peak height... 7-2 Height / Area... 7-2 Baseline width... 7-3 Width... 7-3 Peak sigma (Standard deviation)... 7-3 Peak skew (Asymmetry)... 7-4 Peak kurtosis (Peak top flatness)... 7-4 iv Table of Contents

Peak quality criteria... 7-5 Index... Index-1 Table of Contents v

vi Table of Contents

1 Peak Response and Peak Ratio There are two main methods of calculating peak response value: External Standard and Internal Standard. Both are based on the Absolute Response calculated for a peak. Contents: Topic Page Absolute Response 1-2 External Response 1-2 Internal Response 1-3 Primary/Secondary Peak Ratio 1-3 1-1

Absolute Response Absolute Response is based on either peak area or height and the selected combination of compound primary and secondary peaks. The examples below are shown for peak area. Compound response primary This is the default if no secondary trace is specified. Abs.Resp. = Primary Area Compound response secondary Abs.Resp. = Secondary Area Compound response both Abs.Resp. = Primary Area + Secondary Area Primary Area is the area of a peak calculated by peak detection on the primary chromatogram trace. Secondary Area is the area of a peak calculated by peak detection on the secondary chromatogram trace. External Response Peak Response = Abs.Resp Where Abs.Resp is the absolute response of a peak as calculated in Absolute Response on page 1-2. 1-2 Peak Response and Peak Ratio

Internal Response Peak Response Abs.Resp Amount = ------------------------------------------------------- I Abs.Resp I Abs.Resp is the absolute response of a peak as calculated in Absolute Response on page 1-2. Amount I is the given amount of the Internal Standard in the sample. Abs.Resp I is the area of the internal standard peak as calculated in Absolute Response on page 1-2. Primary/Secondary Peak Ratio If primary and secondary traces are specified, the ratio of peaks detected on each trace is calculated as shown below. Ratio is based on either peak area or height: the example below is shown for peak area. Primary Peak Area Ratio = ---------------------------------------------------------- Secondary Peak Area Primary Peak Area is the area of a peak calculated by peak detection on the primary chromatogram trace. Secondary Peak Area is the area of a peak calculated by peak detection on the secondary chromatogram trace. Absolute Response 1-3

1-4 Peak Response and Peak Ratio

2 Calibration Curve Calculations MassLynx can fit several types of calibration curve. Contents: Topic Page Weighted Calibration Curves 2-2 Include Origin 2-2 Average RF 2-2 Linear 2-3 Quadratic and Higher Order Curves 2-4 2-1

Weighted Calibration Curves Calibration points used when fitting curves can be given a weighted importance: the larger the weighting the more significant a point is treated when the curve is fitted. Weighting (w i ) of i th calibration point is calculated using one of the following. All w i are set to 1 for no weighting. 1. 2. 3. w i w i w i 4. w i = x i y i is Y value (response) of i th calibration point. x i is X value (concentration) of i th calibration point. Include Origin If Include Origin is selected as a calibration curve type, an extra point with zero concentration and response is used in the regression. The extra point has the same weighting as the lowest calibration standard unless Force Origin is set, in which case the origin will always be included as a point directly on the calibration curve. Average RF = = = 1 y i 2 y i 1 x i 2 The calibration curve formed is linear passing through the origin with a gradient equal to the average response values of the calibration points. Average RF = Swy ---------- Sw y i x i Swy = w ---------------- i 2-2 Calibration Curve Calculations

Sw = Σw i Linear y i is Y value (response) of i th calibration point. x i is X value (concentration) of i th calibration point. w i is weighting of i th calibration point, all set to 1 for no weighting. The calibration curve is formed by fitting a line using linear regression to a set of calibration points. Gradient = Swxy -------------- Swxx Intercept = y w,mean Gradient x w,mean Σy y i w i w,mean = -------------------- Σw i Σx x i w i w,mean = -------------------- Σw i Swxy = Σ( x i x w,mean ) ( y i y w,mean ) w i Swxx = Σ( x i x w,mean ) 2 w i y i is Y value (response) of i th calibration point x i is X value (concentration) of i th calibration point w i is weighting of i th calibration point, all set to 1 for no weighting. If Force Origin is selected, a line with zero intercept is fitted. Σx Gradient i y i w = ------------------------------- i Σx i 2 w i Weighted Calibration Curves 2-3

Quadratic and Higher Order Curves MassLynx uses a general Least Squares Fit algorithm to regress a polynomial of any order against the calibration points. The method used is outlined below. Polynomial regression can be described as the fitting of m independent variables (Xj, j = 0 to m-1) to a single dependent variable y. In other words: y = Xb + e y is the n x 1 vector containing the n y values (y i ). X is the n x m matrix of x values, (x j i ). b is the m x 1 vector of regression coefficients (b i ). e is the n x 1 vector of residuals from the fit to each y i value. The familiar least squares solution for the regression coefficients is given by: b = ( X X) 1 X y -1 indicates matrix inverse ' indicates matrix transpose The above equation can then be solved using Gauss-Jordan elimination. To implement weighted regression X and y are first multiplied by a diagonal n x n matrix P (in other words, X becomes PX and Y becomes PY), before the above equation is solved. Where each element (p ij ) of P is given by: p ij = w 1/ 2 i for i =j p ij = 0 for i < > j w i is weighting of i th calibration point, all set to 1 for no weighting. 2-4 Calibration Curve Calculations

3 Peak Amount Calculations Contents: Topic Page User Specified Response Factor 3-2 Average RF Calibration Curve 3-2 Linear Calibration Curve 3-2 Quadratic and Higher Order Calibration Curves 3-3 User Parameters 3-3 3-1

User Specified Response Factor If a user response factor if selected within the quantitation method, calibration curves are not used. The following calculation is performed to obtain peak amounts: Amount -------------------------------------------- Peak Response Response Factor Peak Response is the response value calculated for a peak. Response Factor is the user-entered response factor for that compound. Average RF Calibration Curve Amounts are calculated using an Average RF calibration as follows: Amount Peak Response is the response value calculated for a peak. Average RF is the average response factor calculated for a set of calibration points. Linear Calibration Curve = = Peak ---------------------------------------- Response Average RF Amounts are calculated using a linear calibration as follows: Peak Response Intercept Amount = ------------------------------------------------------------------------ Gradient Peak Response is the response value calculated for a peak. Intercept is the intercept calculated for the linear calibration. Gradient is the gradient calculated for the linear calibration. 3-2 Peak Amount Calculations

Quadratic and Higher Order Calibration Curves Amounts are calculated by solving the following equation using the Newton-Raphson Method: Peak Response = P(Amount) Peak Response is the response value calculated for a peak. P( ) is the polynomial function calculated for a set of calibration points. User Parameters User parameters can be used to multiply or divide the final quantitation results. These factors are entered per sample in the Sample List. If a factor is not specified, or is zero, it is assumed to be one. Final Amount Amount User Factor 1 User Factor 2 User Factor 3 = ---------------------------------------------------------------------------------------------------------------------------------------------------------- User Divisor 1 The User Peak Factor is entered per compound in the Quantify Method. Final Amount = Amount User Peak Factor Rules: User multiplication and divisor factors are not applied to standard samples. In MassLynx 3 onwards, Sample List fields can be renamed to user requirements. When converting from a pre MassLynx V3.0 sample list the following field mappings occur: 'Initial Amount' becomes 'User Divisor 1' 'User Factor' becomes 'User Factor 1' 'Dilution Factor' becomes 'User Factor 2' 'Extract Volume' becomes 'User Factor 3' User Specified Response Factor 3-3

3-4 Peak Amount Calculations

4 Calibration Curve Statistics Contents: Topic Page Coefficient of Determination 4-2 Curve Correlation Coefficient 4-2 RRF mean, SD, and %RSD for Average RF curves 4-3 4-1

Coefficient of Determination The coefficient of determination is calculated for a regressed calibration curve. In the case of a linear curve it is equivalent to the square of the correlation coefficient and is reported as such. For each data point a value of y (y i,pred ) can be predicted from the calibration curve at the position x i. For each data point a residual between the actual and predicted y value can be calculated as (y i - y i,pred ), and the weighted residual sum of squares (RSS) can be calculated as: RSS = Σ( y i y i,pred ) 2 w i where w i is the weighting of the i th calibration point. The total variation in the data is reflected in the weighted corrected sum of squares (CSS), calculated as: CSS = Σ( y i y w,mean ) 2 w i where y w,mean is the weighted mean value of y Σy y i w i w,mean = -------------------- Σw i The model sum of squares (MSS) is the portion of the total variation accounted for by the regression: MSS = CSS RSS The coefficient of determination (R 2 ) is the proportion of the variation accounted for by regression, and is given by the ratio of the model sum of squares to the corrected sum of squares: R 2 = MSS / CSS = (CSS RSS) / CSS Curve Correlation Coefficient In the case of linear curves, the square of the correlation coefficient (r) is equivalent to the coefficient of determination (see Coefficient of Determination on page 4-2) and is reported as such. 4-2 Calibration Curve Statistics

RRF mean, SD, and %RSD for Average RF curves The calculation of RRF mean, SD, and %RSD are detailed in the following equations: Mean RRF RRF i = -------------------- N N = Abs.Response -------------------------------------- i Conc ---------------------------------------------- N i N ( RRF i Mean RRF ) 2 SD N RRF = ---------------------------------------------------------- N 1 SD RRF %RSD RRF = ----------------------- 100% Mean RRF Coefficient of Determination 4-3

4-4 Calibration Curve Statistics

5 Totals Compounds Totals compounds are formed from groups of peaks. These can include Named peaks that have been identified by other method compounds and Unnamed peaks which have not been identified. Compounds are identified as being a part of a Group by a shared group name. This section details calculations performed specifically for Totals compounds. Contents: Topic Page Peak Response 5-2 External Response 5-2 Internal Response 5-2 Calibration Curve Calculations 5-3 Peak Amount Calculations 5-3 5-1

Peak Response The responses of all peaks that are part of the Totals group are calculated individually. External Response Refer to External Response on page 1-2. Internal Response Calculated Internal Response of an unnamed peak is based upon the average response of the Internal Standards of the named compounds in the Group. Peak Response = Abs.Resp -------------------------------------------------- Ave.Named.Resp IS Abs.Resp = Absolute response of Unnamed peak as calculated in Absolute Response on page 1-2. Ave.Named.Resp IS = Average response of group Named peaks Internal Standards. = ( Σ N ( AbsResp ISn Amount ISn )) N N = Number of found Named compounds in group. n = Named compound index. AbsResp ISn = Absolute response of the n th Named Internal Standard peak as calculated in Absolute Response on page 1-2. Amount ISn = Given amount of the n th Named Internal Standard in the sample. 5-2 Totals Compounds

Calibration Curve Calculations The calibration curve for a Totals compound is formed by averaging the calibrations of the Named compounds in the group. The Totals curve is used to calculate the concentration of the Unnamed peaks in the group. Each coefficient of the Totals curve is calculated as follows: C i = ( Σ N C i,n ) N N is the number of Named compounds in the group. C i is the i th coefficient of the Totals curve. C i,n is the i th coefficient of the n th Named compound in the group. Peak Amount Calculations Totals concentration is the sum of the concentrations of the selected peaks that make up the group. Peaks summed can be specified as only Named peaks, only Unnamed peaks, or both. The concentration of each individual peak in the group is calculated as described in Peak Amount Calculations on page 3-1. Peak Response 5-3

5-4 Totals Compounds

6 Limits of Detection (LOD) and Limits of Quantitation (LOQ) The calculation of the LOD and LOQ for a compound is dependent upon the noise contained within the chromatogram trace used to quantify that compound. Contents: Topic Page Chromatogram Noise Calculation 6-2 Chromatogram Area Noise 6-3 Response Value for Noise 6-3 LOD and LOQ Concentrations 6-4 LOD and LOQ Flags 6-4 T-LOD and T-LOQ Concentrations 6-4 6-1

Chromatogram Noise Calculation Chromatogram noise is calculated as the standard deviation of the noise in the trace (effectively a height), which is multiplied by a user specified factor. Chromatogram noise may be calculated automatically or manually. Manual noise measurement To select this method, the user must specify an RT range of the trace which contains noise only. The resulting noise value is the standard deviation of the data in this region, multiplied by the User Factor. Tip: An RT entry of 0.0 may be used to signify either the start or end of the trace if input as the start or end RT respectively. If both the start and end RT are set to 0.0, automatic noise measurement (see page 6-2) will be initiated. Noise Height = SD selected data User Factor ( x i Mean x ) SD N x = ---------------------------------------- N 1 Automatic noise measurement This method is initiated by selecting the entire range of the trace for noise measurement (that is, start and end RT equal 0.0). An algorithm is used to determine which regions of the trace consist only of noise, the result being the standard deviation of these regions multiplied by the user specified factor. Rule: The noise detection algorithm relies on a relationship between the median difference between adjacent points and standard deviation which is specific to gaussian deviates. This approximation provides good results when used with raw chromatogram data, but not for smoothed data. As a result, the algorithm measures the noise in the raw trace, to which a scaling factor is then applied to produce the effect of the current smoothing parameters. This smoothing correction is specific to the effect of mean smoothing on gaussian deviates, explained below. 6-2 Limits of Detection (LOD) and Limits of Quantitation (LOQ)

The accuracy of this method is affected: when Savitzky Golay smoothing is used to a lesser extent, when the noise regions are not gaussian in nature. SD meansmoothed This equation describes the relationship between the standard deviation of raw and mean smoothed data (for gaussian deviates). Iterations is the number of smooths applied to the raw data and Winsize is the size of the smoothing window. MassLynx requires a half window size to be entered which has the relationship with Winsize described above. A multiplication factor of 3 is commonly used as, for gaussian deviates, this corresponds to a range within which 95% of the data will lie. Chromatogram Area Noise The noise value is effectively a measure of noise height: an area value may be calculated that is equivalent to the calculated noise. The Compound s IS peak is used for this: the ratio of height to area of the corresponding IS peak is calculated, and this is applied to the noise value if the corresponding noise area is required. Response Value for Noise = SD raw data -------------------------------------------------------------------- Winsize 0.5 Iterations 0.25 Winsize = ( 2 WinHalfSize) + 1 The absolute noise values are converted into equivalent response values. If compounds are being quantified using Heights, use the original noise values; if using Areas, use the Area noise values. Calculate an absolute noise response for the compounds based upon the method Secondary Parameters Compound response, Primary, Secondary or Both (in which case sum the noise values from both traces). If a Secondary trace is not specified the Primary is always used. Calculate a noise response value from the absolute noise response by applying the IS response, in the same way as if it were a normal peak. Chromatogram Noise Calculation 6-3

LOD and LOQ Concentrations The LOD and LOQ concentrations are calculated by applying the compound s calibration curve to the noise response to obtain a value for the concentration, which is then multiplied by user specified factors for LOD and LOQ respectively. The LOD and LOQ factors are specified as part of the QuanLynx or TargetLynx method and are set for all compounds. The default values are 3 and 8 respectively. Resulting LOD and LOQ concentrations should be stored in Peak Record and be available for display in the Summary. If no value is available output should be blank. LOD and LOQ Flags These flags indicate if the calculated concentration of a compound falls outside the LOD or LOQ concentration threshold. If a compound concentration exceeds the threshold the flag is Yes. If it is less than the threshold the flag is No. T-LOD and T-LOQ Concentrations The Toxic LOD concentration is calculated by multiplying the Toxic Equivalence Factor by the Limit Of Detection. The Toxic LOQ concentration is calculated by multiplying the Toxic Equivalence Factor by the Limit Of Quantification. Resulting T-LOD and T-LOQ concentrations should be stored in the Peak Record and be available for display in the Summary. If no value is available output should be blank. 6-4 Limits of Detection (LOD) and Limits of Quantitation (LOQ)

7 Peak Statistics Contents: Topic Page Peak height 7-2 Height / Area 7-2 Baseline width 7-3 Width 7-3 Peak sigma (Standard deviation) 7-3 Peak skew (Asymmetry) 7-4 Peak kurtosis (Peak top flatness) 7-4 Peak quality criteria 7-5 7-1

Peak area The peak area is calculated by summing the areas of each of the segments that make up the peak. Y2 Y3 Y4 Y1 X0 X1 X2 X3 X4 Xn Xn is the retention time (mins) at which sample n was taken. Yn is the intensity above the baseline for sample n. The area of each individual segment is calculated using the trapezium rule. ( Y Area i + 1 + Y i ) i = --------------------------- ( X 2 i + 1 X i ) Total Area = ΣArea i n 1 The advantage of using this method is that the calculated peak area is less dependent on the sampling rate across the peak. Peak height The peak top is the retention time with the largest absolute intensity. Peak height is calculated as the distance between this largest intensity and the peak baseline at that retention time. Height / Area The peak height divided by the peak area. 7-2 Peak Statistics

Baseline width Width The difference, in seconds, between the end of the peak baseline and the start of it. Peak width measured in seconds at 50% of the peak height above the baseline. The value displayed is an approximation calculated as follows. The peak width at the inflection points is actually calculated this is known as the peak width at 4 σ (or 4 sigma). The majority of peaks are Gaussian in shape above the inflexion points, since the most variability is at the base of the peak. The peak width at half height is calculated from the equation: Width at half height = 2 2ln(2) σ = 2.35482 σ Width at inflection points = 4 σ Peak sigma (Standard deviation) This is the standard deviation of the data, calculated as the square root of the variance. The weighted variance is calculated as: -- 1 d w i ( x i x) 2 x i is the retention time value for observation i. w i is the intensity associated with x i. d = Σw i x is the weighted mean of x whose formula is: Σw i x i Σw i Peak area 7-3

The standard deviation is calculated as the square root of the variance: σ( x 1...x n ) = Var( x 1...x n ) The variance and standard deviation both require at least two non-missing observations. Peak skew (Asymmetry) The peak skewness characterizes the degree of asymmetry of a distribution around its mean (that is, a measurement of the tendency of the deviations to be larger in one direction than in the other). It is non-dimensional. It is a pure number that characterizes only the shape of the distribution, defined as follows: 1 = -- n ( ( x i x) σˆ j) 3 1 = -- n w 3 2 i (( x i x) σˆ ) 3 x is the weighted mean of x. x i is the retention time value for observation i. w i is the intensity associated with x i. σ is the distribution s weighted standard deviation defined above. A positive value of skew signifies a distribution with an asymmetric tail extending out towards more positive x; a negative value signifies a distribution whose tail extends out towards more negative x. Skewness requires at least three non-missing observations. Peak kurtosis (Peak top flatness) Peak kurtosis is a non-dimensional quantity. It measures the relative peakedness or flatness of a distribution, relative to a normal distribution. A distribution with positive kurtosis is termed leptokurtic; the outline of a very peaked mountain is an example. 7-4 Peak Statistics

A distribution with negative kurtosis is termed platykurtic; the outline is topped like a plateau. Distributions that are neither leptokurtic nor platykurtic are termed mesokurtic. Weighted kurtosis is computed as follows: 1 = -- n ( ( x i x) σˆ i) 4 3 1 = -- n w 2 i (( x i x) σˆ ) 4 3 σ 2 i is σ 2 w i x is the weighted mean of x. x i is the retention time value for observation i. w i is the intensity associated with x i. σ is the distribution s weighted standard deviation defined above. The subtraction of 3 in the above expressions normalizes the kurtosis of a Gaussian distribution to zero. Kurtosis requires at least four non-missing observations. Peak quality criteria The peak statistics described above are measured against a selection of peak quality criteria (defined in the Method Editor) to generate Pass or Fail status in Peak Quality and a description of any failure in Peak Quality Description. Peak area 7-5

7-6 Peak Statistics

Index Symbols %RSD 4-3 A Absolute Response 1-2 asymmetry 7-4 Average RF 2-2, 4-3 calibration 3-2 B baseline width 7-3 C calibration curves 2-1 calculations 5-3 Chromatogram noise 6-2 coefficient of determination 4-2 concentration thresholds 6-4 corrected sum of squares 4-2 E External Response 1-2, 5-2 F Flags LOD and LOQ 6-4 G Gaussian 7-3, 7-5 Gauss-Jordan elimination 2-4 Gradient 3-2 H Higher Order curves 2-4, 3-3 I Include Origin 2-2 Intercept 3-2 Internal Response 1-3, 5-2 Internal Standards 5-2 K kurtosis 7-4 L Least Squares Fit 2-4 Limits of Detection 6-1 Limits of Quantitation 6-1 linear 2-2, 2-3 calibration 3-2 LOD 6-1 LOQ 6-1 M model sum of squares 4-2 N Named peaks 5-1 Newton-Raphson Method 3-3 Noise measurement 6-2 P peak area 7-2 height 7-2 kurtosis 7-4 named 5-1 response 1-1, 3-2, 5-2 skewness 7-4 top flatness 7-4 unnamed 5-1 Peak Amount Calculations 5-3 peak quality criteria 7-5 Primary Peak Area 1-3 Index-1

Q Quadratic curves 2-4, 3-3 quantitation method 3-2 results 3-3 R ratio primary and secondary peaks 1-3 regressed calibration curve 4-2 residual sum of squares 4-2 Response Factor 3-2 retention time 7-2 RRF mean 4-3 S Savitzky Golay 6-3 SD 4-3 Secondary Peak Area 1-3 standard deviation 7-3 sum of squares corrected 4-2 model 4-2 residual 4-2 T Totals compounds 5-1 U Unnamed peaks 5-1 User parameters 3-3 V variance weighted 7-3 W Weighted Calibration Curves 2-2 weighted variance 7-3 Index-2