Ballot for Revised Annex R - API Guidelines for Use of Single Technology Matrix

Transcription

1 T: Cc: API Lubricants Grup Lubricants Grup Mailing List API Ballt fr Revised Annex R - API Guidelines fr Use f Single Technlgy Matrix On Nv. 14, 2018 the Lubricants Standards Grup (LSG) discussed the BOI/VGRA, Annex R/STM Wrk Grup prpsed revisin f Annex R - API Guidelines fr Use f Single Technlgy Matrix. The Discussin was a summary f significant changes with a cpy f the Revised Annex R. The significant revisins t Annex R were summarized in a presentatin: API 1509 Annex R Dcument Clarificatin - Nvember 2018 Overview (Attachment 1). The Revised Annex R was prvided in dcument: Annex R - Edits Made After June 28 Meeting API Guidelines fr Use f Single Technlgy Matrix (Attachment 2) After review and discussin, the LSG agreed by vice vte t Ballt Revised Annex R - API Guidelines fr Use f Single Technlgy Matrix. Mtin Ballt the revised versin f Annex R as explained in Attachment. Mtin by: Dan Pridemre Secnd by: Muibat Gbadamsi/Calumet Vice Vte: Fr: 14 Against: 0 Abstain: 0 Mtin Passed Lubricants Grup Members shuld use the API Ballt System t cast their vte and make cmments. The Ballt Link is: The Lubricants Grup Member vtes will be cunted, and all received cmments reviewed and cnsidered befre the ballt results are final. Nn-Lubricants Grup Members shuld cmment n the Ballt Mtin using the Ballt system. The Ballt Link is: All cmments n the Ballt Mtin will be reviewed befre the ballt results are final. Due t the cmplexity f the Revised Annex R - API Guidelines fr Use f Single Technlgy Matrix this ballt will be pen fr apprximately 60 days. Ballt will clse n February 11, All Vtes and/r Cmments must be finalized by the clse date. Page 1 f 1

2 Attachment 1

3 API 1509 Annex R Dcument Clarificatin - Nvember 2018 Overview Cmmunicatin t API Lubricants Standards Grup B W Schwab Nvember 14,

4 Annex R : API Guidelines fr Use f Single Technlgy Matrix Revised dcument was the result f numerus cmpanies wh actively participated in the wrk grup : Aftn Chemical API Chevrn ExxnMbil Infineum Lubrizl KleenPerfrmance Neste Oil Ornite Petr-Canada Shell SK Ttal Valvline 2

5 Annex R : Clarificatin Activity BOI/VGRA Task Frce cmmissined the STM Wrk Grup in 2013 t review and update Annex R WG members are active users f Annex R STM WG has met regularly thrughut the past 5 years and methdically wrked thrugh Annex R t: Clarify the riginal intent Imprve readability fr all users After thrugh review, the WG shared the final draft n July 18 and infrmed the BOI/VGRA TF at August 29 meeting that the clarificatin dcument is ready fr ballting 3

6 Annex R : API Guidelines fr Use f Single Technlgy Matrix Wrk Grup has revised Annex R t make it a mre readable dcument Draft is intended t clarify, it is nt intended t change the riginal intent f the dcument Cannt be cmpared line by line 4

7 Annex R : STM July 2018 Draft Recmmend t read the draft dcument fr meaning Des it tell yu what yu already knw? Des it supprt yur understanding f STMs? Rerganized the sectins t have general infrmatin up frnt Tests cvered by STM had been listed in the back in R.6 Multiple Technlgy Matrix left ut; build with Mdified Technlgies Added additinal definitins The statistics infrmatin was kept in sectin R.2 R.2.1 presents clearer scpe fr STM Added mre details n the analysis requirements Sectins R.3 thrugh R.6 were rerdered fr better flw Expanded the Summary f Requirements t include Guidelines and Uses (nw R.4) Added mre details t the examples in R.5 and R.6 5

8 Annex R : Highlights f Clarificatins t Accmpany Ballt Dcument cntents are rerdered t have general infrmatin at the frnt f the dcument and the statistics language and examples at the back f the dcument Multiple Technlgy Matrix was remved frm the language f Annex R Wrk Grup felt it was nt being used; remving it did nt change the way STM is practiced The minimum number f tests required fr an STM is based n the number f identified Base Oil Prperties f Interest + 2 additinal tests. Currently Annex R identifies 5 prperties fr accepted STM tests. It was agreed that fr the current accepted STM test types, Base Oil Saturates and Base Oil Sulfur wuld be drpped as Base Oil Prperties f Interest when the STM cntain nly Grup II and/r Grup III base ils, thus reducing the number f Base Oil Prperties f Interest t 3 and the minimum number f tests required fr thse STMs t 5 tests. This reflects a change fr sme cmpanies that had been using either 6 r 7 tests depending n the interpretatin f the riginal Annex R The change des nt impact STM validity 6

9 Annex R : Objectives f Clarified Annex R Intended t make Annex R mre understandable t the general user Trying t imprve the readability f the dcument fr the statistician and nn-statistician user Prvide guidance t the new statistician being asked t cnstruct and/r extract data fr an STM Have Industry feel cmfrtable with Annex R t cnsider applying the STM apprach fr ther test types such as the Sequence IIIH 7

10 Annex R : STM July 2018 Draft Next Steps Mtin: Issue a ballt with this accmpanying infrmatin t adpt the edits t API 1509 Annex R, July 2018 draft dcument 8

11 Mtin t Ballt Ballt the revised versin f Annex R as explained in Attachment. Mtin by: Dan Pridemre Secnd by: Muibat Gbadamsi/Ryal Purple Hand Vte Results: Fr = 14 Against = 0 Abstain = 0 9

12 Attachment 2

13 June 28, 2018 Phenix Meeting Annex R - Edits Made After June 28 Meeting API Guidelines fr Use f a Single Technlgy Matrix R.1 General Principles The Single Technlgy Matrix (STM) apprach is based n a specific technlgy frm a single supplier; the statistical mdel develped frm a Single Technlgy Matrix applies nly t the technlgy used in that STM. 1 Currently STM is nly applicable t select Sequence III tests (Sequence IIIF, IIIFHD, IIIG, and IIIGA) but culd be adapted fr ther engine tests if the industry agreed t d s. T incrprate future tests, it wuld be necessary t understand the key physical and chemical prperties that influence a test s the defining Base Oil Prperties f Interest culd be determined. Please refer t sectin R R.1.1 Intrductin The STM refers t the set f test results n a specific technlgy fr a specific API Perfrmance Categry generated n test ils with Base Oil and Finished Oil prperties that span an perative range. The tested il prperties and engine test results are analyzed t establish a predictive mdel fr that range f Base Oil and Finished Oil prperties. The mdel is used t generate a Predicted Test Result fr a Candidate Oil. If the predictin meets the statistical requirements as utlined in Sectin R.2.2, the Predicted Test Result is used in lieu f an engine test result t supprt API license claims fr the Candidate Oil. An STM Predicted Test Result shuld nt be used t verride an actual failing test result n a Base Oil with this technlgy. The Predicted Test Result and STM supprt dcumentatin are reprted in the Candidate Data Package. The set f tests that are currently cvered by the STM include the fllwing: IIIF IIIFHD IIIG IIIGA R.1.2 Definitins In rder t better understand the develpment and applicatin f an STM, these key terms are required. R A Single Technlgy as designed fr use in an STM is a single additive package (DI) at a cnstant treat rate, with a single viscsity mdifier, and in a single viscsity grade. 1 The Base Oil Interchange/Viscsity Grade Read Acrss (BOI/VGRA) guidelines in API 1509 are develped thrugh industry cnsensus. Each guideline is generally derived using the minimum quality and quantity f data principle. This principle requires that three technlgies frm at least tw cmpanies agree n the characteristic behavir f the base stck slates and/r SAE viscsity grade(s). This prcess has the benefit f industry cnsensus and public display f data. Prir t the adptin f API Guidelines fr use f a Single Technlgy Matrix (STM) in 2002, Base Oil Interchange (BOI) fr the Sequence III test had been prpsed. This prpsal was based n industry experience and extensive data in a large variety f Base Oils. Discussins at the Administrative Guidance Panel (AGP) level indicated that the Sequence III test wuld nt be accepted as a candidate fr typical BOI but wuld adpt STM fr the Sequence III test. R A Mdified Technlgy used in building an STM is a frmulatin cntaining a mdificatin. The allwed mdificatins are described in and Fr each Mdified Technlgy in the STM, ne additinal passing test is required by adding ne unique Base Oil. This cncept is different than that f a minr frmulatin mdificatin as defined by ACC Cde f Practice Appendix H. 1

14 June 28, 2018 Phenix Meeting R A Single Technlgy Matrix cnsists f a grup f data meeting the criteria utlined in Annex R, Sectin R.2. The test results in the matrix are based n a Single Technlgy as described in R and, if necessary, a Mdified Technlgy as described in R R A Base Stck is defined in Annex E, Sectin E R A Base Oil is defined in Annex E, Sectin E A Base Oil used in an STM can cnsist f a single base stck r a blend f base stcks. The Base Oil can be cmprised f base stcks frm ne r mre base stck slates. The Base Oil used in a Single Technlgy Matrix (in additin t the definitin in E.1.2.3) is als defined by the relevant Base Oil Prperties f Interest. A Base Stck Slate is defined in Annex E, Sectin E R The Base Oil Prperties f Interest are Base Oil r Finished Oil prperties decided by the API BOI/VGRA Task Frce t be meaningful and influential t engine test perfrmance fr the test cvered by STM. The Base Oil Prperties f Interest are test type specific, defined in sectin R.5, and cnsist f ne r mre f the fllwing: Base Oil Saturates (ASTM D2007) Base Oil Sulfur (API Apprved Tests frm Annex E, Table E-1) Base Oil Viscsity at 100 C (ASTM D445) Base Oil Viscsity Index (ASTM D2270) Finished Oil Nack Vlatility (ASTM D5800) R A Spread Requirement is a stipulatin n the Base Oil Prperties f Interest in the STM that facilitates a symmetric spread in thse prperties. Fr each Base Oil Prperty f Interest that has t meet the spread requirement (refer t table R.5.0), the number f Base Oils n either side f the mean f all Base Oils in the matrix must be within 1 f equal in rder t satisfy the spread requirements. The Base Oils at the mean shuld be cunted as zer. The guiding principle fr the spread requirement fr the selectin f Base Oils in the STM is t avid skewing the analysis by clustering the data at an extreme r at the mean. The number f Base Oils at the mean needs t be equal t r less than the number f Base Oils n either side f the mean. Fr the Sequence IIIF, IIIFHD, IIIG, and IIIGA tests, the Spread Requirement is nly applied t Base Oil saturates and viscsity index (VI), unless the Spread Requirement Waiver fr Saturates (per sectin R.1.2.8) applies and then nly VI needs t meet the Spread Requirement. When perfrming the spread calculatins fr an STM, all raw data and data means must be prperly frmatted accrding t the specific ASTM test prcedure listed in R and using the ASTM E29 standard practice fr runding. Fr saturates, there is n frmal statement in the ASTM D2007 standard n the number f significant figures/runding. As a result, the recmmended practice is t rund saturates t the nearest tenths place. VI is runded t whle numbers. The runding fr VI is applied t each Base Oil in the matrix and the verall mean f the Base Oils in the matrix. A Base Oil may be repeated but the Base Oil Prperties f Interest nly cunt nce in the Spread Requirement calculatins. A Base Oil repeat may be the result f MTEP, utliers, etc. Given that runding is invlved, when cunting the number f Base Oils n either side f the mean, a Base Oil Prperty f Interest equal t the mean shuld be cunted as zer. The Base Oil Prperties f Interest reprted and used in the STM / spread requirement analysis shuld accurately reflect the batches f Base Stcks used in the STM. This can be accmplished by ne f three ways and the selected 2

15 June 28, 2018 Phenix Meeting methd shuld be dcumented. The first is by a direct analysis f the Base Oil Prperties f Interest fr the blended Base Oil. The secnd is by calculatin f the Base Oil Prperties f Interest frm values assciated with the individual base stcks cmprising the Base Oil. The third methd applies if ne is relying n histrical Base Oil/stck data t create an STM with missing prperties. With the third methd, an effrt shuld be made t accurately reflect the prperties used with an explanatin prvided. R The Spread Requirement Waiver fr Saturates allws fr the eliminatin f any spread requirement fr saturates, if, and nly if, every Base Oil in the matrix is cmprised f base stcks belnging t API Grup II, Grup III, and/r Grup IV, because saturates is nt cnsidered an imprtant prperty f interest fr these grups f base stcks. The Spread Requirement Waiver fr Saturates currently applies nly t the Sequence IIIF, IIIFHD, IIIG, and IIIGA tests. R Frmulatin Mdificatins are described in the American Chemistry Cuncil (ACC) Cde f Practice. Mdificatins are allwed in the develpment f an STM but require an additinal Base Oil and test result fr a given mdificatin. These mdified frmulatins are called Mdified Technlgies. R A Viscsity-Grade Change is allwed within an STM in the directin f previusly apprved API 1509 Viscsity Grade Read Acrss Guidelines. Such a change is allwed in the develpment f an STM but requires an additinal Base Oil and test result. The STM can then nly supprt the least difficult viscsity grade tested in the STM as defined by API 1509 Viscsity Grade Read Acrss Guidelines (Annex F). R The Predicted Test Result frm the STM mdel can be used in lieu f an actual engine test result prvided that all STM requirements utlined in Annex R are met. The Predicted Test Result shuld nt be used t verride a failing test result. The Predicted Test Result shall be cmprised f the maximum treat level f all Mdified Technlgies in the least difficult viscsity-grade tested in the STM. The least difficult viscsity-grade is as defined in API 1509 Annex F. The predicted result and STM dcumentatin is prvided t the Marketer in the Candidate Data Package (CDP). R A Passing Test Result used t build the STM can be btained frm a single test r by using the apprpriate Multiple Test Evaluatin Prcedures (MTEP) utlined in ACC Cde f Practice Appendix F. R A Sequence IIIGA Matrix Requirement is that the fresh il MRV f the candidate il, blended t the same viscsity grade, is equal t r less than the fresh il MRV f at least ne f the passing ils in the matrix, within the precisin f the test. R An Outlier is a test result in which the abslute value f the Studentized Residual fr that bservatin frm the analysis is at the ne-sided 97.5 th percentile, r beynd, n a Student t distributin. Outliers must be shwn, but may be remved frm the STM analysis and Spread Requirements. See sectin R.2.4. R A Candidate Oil is a Base Oil blended with the final technlgy in a specific viscsity grade fr a specific perfrmance test in a specific API perfrmance categry whse perfrmance is being predicted by an existing STM mdel. The Candidate Oil s relevant Base Oil Prperties f Interest shall fall within the defined ranges f the STM. R.2 Scpe and Criteria fr a Single Technlgy Matrix R.2.1 Scpe and Criteria fr STM Data and Matrix The STM apprach is used t establish a Predicted Test Result fr a Candidate Oil blended with a specific technlgy in a specific viscsity-grade fr a specific perfrmance test in a specific API perfrmance categry. A predicted result frm the STM that meets the requirements f Annex R can be used in lieu f an engine test result. It is pssible that the predictin mdel may be applicable t mre than ne API perfrmance categry and different limits may apply depending n the categry. Therefre, a separate STM analysis must be perfrmed and dcumented fr each perfrmance categry in which a Predicted Test Result is desired. 3

16 June 28, 2018 Phenix Meeting The STM matrix data must be develped using a Single Technlgy as described in R.1.2.1, and, if necessary, a Mdified Technlgy as described in R The STM apprach nly cvers Grup I, II, III and IV base stck grups. The Predicted Test Result must be fr the final Mdified Technlgy frm the STM matrix data. All tests dcumented in an STM shall be ASTM peratinally valid ACC registered tests. In an STM, the minimum number f unique passing Base Oils is determined by the number f Base Oil Prperties f Interest relevant t the mdel plus 2. Fr each Mdified Technlgy the number f unique Base Oils increases by 1. Fr the tests fr which STM applicatin has been defined (IIIF, IIIFHD, IIIG, IIIGA), the default number f Prperties f Interest is 5 and the minimum number f Base Oils is equal t 7. If the Base Oils are cmprised f API Grup II, III, IV, then the number f Prperties f Interest is 3 (excludes saturates and sulfur) and the minimum number f Base Oils is equal t 5. The minimum number f unique passing Base Oils shall nt be less than five. Base Oil Prperties f Interest are determined by the API BOI/VGRA Task Frce n a per test-type basis; Base Oil Prperties f Interest are listed in R The range f Base Oil Prperties f Interest, defined by the maximum and minimum f the data used t develp the STM mdel, must encmpass the Base Oil Prperties f Interest f the Candidate Base Oil. T extend t a Candidate Base Oil utside this range wuld require at least ne added test n an il with Base Oil Prperties f Interest at r beynd the Candidate Base Oil. A new STM wuld then be develped t include the new test result. Once a Predicted Test Result has been established minr frmulatin mdificatins as per American Chemistry Cuncil Prduct Apprval Cde f Practice (Appendix H and I) and VGRA as per API 1509 Viscsity Grade Read Acrss Guidelines (Annex F) may be applied. R.2.2 STM Analysis Requirements An empirical mdel fr the test results must be develped frm the STM data. If the data set includes ne r mre Mdified Technlgies, the data can be cded as categrical and/r cntinuus variable data. The final frm f the data fr analysis and mdel is at the discretin f the rganizatin develping the STM. Based n the empirical mdel, the Predicted Test Result fr the Candidate Oil must meet the perfrmance specificatin f interest and a cnfidence interval requirement fr the Predicted Test Result must be met as described in R and R R Primary Test The Primary Test is t determine if the entire 95% cnfidence interval fr the predicted mean perfrmance (R.2.3.2) is within the perfrmance specificatin f interest, i.e., the cnfidence interval f the Predicted Test Result, which is based n the data set used in the analysis, des nt extend beynd the pass limit int the fail regin f the specificatin. Passing Pass/Fail Threshld Failing Predicted 95% Cnfidence Interval abve Pass/Fail Limit Threshld R Secndary Test 4

17 June 28, 2018 Phenix Meeting If the Predicted Test Result meets the perfrmance requirement, but the Predicted Test Result Cnfidence Interval (R.2.3.2) crsses the pass/fail threshld int the fail regin, then and nly then can the Secndary Test be applied. The Secndary Test evaluates the width f the Predicted Test Result Cnfidence Interval and cmpares it t the Cnfidence Interval fr a Mean Based n a Single Test Result (R.2.3.1), which is based n the Nrmal Frequency Distributin and the current standard deviatin f the test used in the calculatin f severity adjustments as defined in ASTM Test Mnitring Center Technical Memrandum , Annex C f the LTMS Manual. The width calculated in R cannt be greater than the width calculated in R Althugh the cnfidence intervals must be calculated in the apprpriate transfrmed units, the cmparisn must be made in riginal units. Predicted Engine Test can be used in lieu f an actual engine test. (R Width < R Width) Passing Predicted Test Result Cnfidence Interval Width Based n the Mdel Predictin Predicted Test Result Pass/Fail Threshld Cnfidence Interval Width fr a Mean Based n a Single Test Result Failing Predicted Engine Test CANNOT be used in lieu f an actual engine test. (R Width > R Width) Passing Predicted Test Result Cnfidence Interval Width Base Based n the Mdel Predictin Predicted Test result Pass/Fail Threshld Cnfidence Interval Width fr a Mean Based n a Single Test Result Failing R.2.3 Calculatin f Width f 95% Cnfidence Interval R Industry Cnfidence Interval Width fr a Mean Based n a Single Test Result (CIW1) CIW1 = 2 x Z 0.05 x 5

18 June 28, 2018 Phenix Meeting Where: Z 0.05 = 1.96 = current standard deviatin f the test used in the calculatin f severity adjustments as defined in ASTM Test Mnitring Center Technical Memrandum , Annex C, f the LTMS Manual. This is the shrtcut methd fr calculating the width f the methd 1 cnfidence interval. If a transfrmatin is required, the shrtcut methd cannt be used. The actual cnfidence interval must be calculated fr the predicted result fr the il n the transfrmed scale. This is dne by adding and subtracting Z x frm the Predicted Test 0.05 Result, transfrming the cnfidence limits back, and then subtracting the limits n the riginal scale. R Predicted Test Result Predicted Cnfidence Interval Width based n the Mdel Predictin (CIW2) CIW2 = 2 x t 0.05,df x S x h i Where: t 0.05,df = Student T distributin at the 95% Cnfidence Level with degrees f freedm equal t the degrees f freedm used in the estimate f the Rt Mean Squared Errr (RMSE) S = Rt Mean Squared Errr frm the analysis h i = x (X X) -1 xi 1 i X = the factr matrix x i = a particular factr setting This is the shrtcut methd fr calculating the width f the methd 2 cnfidence interval. If a transfrmatin is required, the shrtcut methd cannt be used. The actual cnfidence interval must be calculated fr the predicted result fr the il n the transfrmed scale. This is dne by adding and subtracting t 0.05,df x S x hi frm the transfrmed predicted result, transfrming the cnfidence limits back, and then subtracting the limits n the riginal scale. R.2.4 Calculatin f the Studentized Residual and Outlier Test Outliers may be remved frm the STM analysis as lng as bth the spread requirement and minimum number f Base Oils requirement are satisfied upn re-evaluatin with the Outlier remved. If an Outlier is remved, then all test related data fr that bservatin must be remved frm the analysis. Hwever, Outliers must still be included in the STM dcumentatin. Nte: while bservatins may be drpped accrding t the Multiple Test Evaluatin Prcedure (MTEP) t determine pass/fail, thse bservatins may nt be drpped frm the STM analysis unless declared an Outlier accrding t R.2.4. Where: e*i ei S(i) hi X xi e*i = ei / (S(i) x ( 1-h1 )) = the Studentized Residual, which is distributed clsely t the Student T distributin. In this applicatin, the ith bservatin fr a test parameter may be declared as an utlier and remved frm the analysis if e*i is greater than the ne sided t0.025,df with degrees f freedm equal t the degrees f freedm used in the estimate f the Rt Mean Squared Errr. = the residual frm the analysis, the actual test result fr the ith bservatin fr a parameter minus the predicted test result fr the ith bservatin fr a parameter. = Rt Mean Squared Errr frm the analysis with the ith bservatin remved frm the analysis = xi (X X) -1 xi = the factr matrix = a particular factr setting R.3 Ntificatin f Single Technlgy Matrix Use t API 6

19 June 28, 2018 Phenix Meeting The Matrix data and analysis must be shwn t the Oil Marketer within the Candidate Data Package. Oil Marketers must ntify API n the EOLCS Applicatin fr Licensure whenever STM data is used t qualify an il frmulatin fr API licensing. The n-line license applicatin asks the questin if STM has been used r nt. When asked t prvide a Frmulatin/Stand Cde n the licensing frm, any ne f the actual test stand cdes frm the STM can be listed n the licensing frm. R.4 Summary f Requirements, Guidelines and Uses fr the Single Technlgy Matrix The requirements, principals and guidelines fr the STM are summarized belw: a. STM is used t establish a Predicted Test Result fr a Candidate Base Oil fr a specific technlgy (see R.1.2.1) in a specific viscsity-grade. Applicable VGRA may be applied t the Predicted Test Result. b. A Predicted Test Result is used in lieu f an engine test result and can be treated as an actual test result in the executin f a prgram. c. The STM apprach nly cvers Base Oils cmprised f base stcks belnging t API Grups I, II, III, and IV. d. The API BOI/VGRA Task Frce reviews perfrmance engine tests fr STM use, defines the Base Oil Prperties f Interest, and recmmends use f the STM t the Lubricants Grup. e. The API Lubricants Grup apprves the Base Oil Prperties f Interest and use f the STM fr perfrmance engine tests recmmended by the API BOI/VGRA Trask Frce. f. The Matrix Data Criteria must be met as defined in R.2.1. g. All tests in the develpment f the STM dataset and analysis must be registered accrding t the ACC Cde f Practice. h. The single technlgy must pass within a single test result r by using the apprpriate Multiple Test Evaluatin Prcedures (MTEP) utlined in ACC Cde f Practice Appendix F fr each Base Oil in the STM fr all relevant test parameters. If nt, a frmulatin adjustment r a Viscsity Grade Change may be made t the Single Technlgy t create a Mdified Technlgy within the STM. The Mdified Technlgy must then pass all remaining Base Oils nt passed using the Single Technlgy, within a single test result r by using the apprpriate MTEP fr all relevant test parameters. The minimum number f unique Base Oils in the STM MUST increase by ne (1) fr every Mdified Technlgy used t demnstrate an acceptable STM. i. Data may nt be drpped frm the STM analysis unless declared an Outlier accrding t R.2.4. Test results drpped fr evaluatin in an MTEP prcedure may nt be drpped frm the STM analysis unless declared an Outlier accrding t R.2.4. Outliers must be shwn in the STM dcumentatin, but may be remved frm the STM analysis and Spread Requirements. j. The width f the 95% cnfidence interval (based n the Student t distributin) fr the predicted mean perfrmance based n the STM mdel cannt be greater than the width f the 95% cnfidence interval (based n the Nrmal Frequency Distributin and the current standard deviatin f the test used in the calculatin f severity adjustments as defined in ASTM Test Mnitring Center Technical Memrandum , Annex C, f the LTMS Manual) fr the mean based n a single test result at the predicted perfrmance level UNLESS the 95% cnfidence interval fr the predicted mean perfrmance (based n the Student t distributin) is within the perfrmance specificatin f interest (see R.2.2). k. API will survey additive cmpanies n a regular basis fr STM data l. STM shuld nt be used t ver-ride a failing test. m. The engine test and the applicable API perfrmance categry in which the test is used must be defined and identified in any rganizatin s STM analysis and dcumentatin. A separate STM analysis must be perfrmed and dcumented fr every single perfrmance categry in which a Predicted Test Result is desired fr the technlgy. n. STM results must be included in ACC Candidate Data Packages. Ntificatin f use f STM data fr API licensure will be present n an Oil Marketer s API License applicatin and must be checked if used. An example is prvided in R.3.. Future engine test data d nt affect previusly established STMs. 7

20 June 28, 2018 Phenix Meeting p. Once an STM is established, minr frmulatin mdificatins are per American Chemistry Cuncil Prduct Apprval Cde f Practice (Appendix H) and VGRA as per API 1509 Viscsity Grade Read Acrss Guidelines (Annex F) may be applied. R.5 Engine Tests Apprved fr STM The fllwing table summarizes the Base Oil prperties f Interest fr each test type apprved fr STMs. As stated previusly in Sectin R.1.2.7, certain Base Oil Prperties are subjected t the Spread Requirement. In Table R.5.0, thse Base Oil Prperties are identified with an X. Nte: The Spread Requirement fr Base Oil Saturates can be waived if every Base Oil in the STM is cmprised f base tcks belnging t API Grup II, Grup III, and/r Grup IV. Base Oil Saturates Table R.5.0 Base Oil Prperties f Interest Base Oil Sulfur Base Oil Viscsity at 100 C Base Oil Viscsity Index IIIF X X IIIF HD X X IIIG X X IIIGB X X Nack Vlatility f the fully frmulated il Passenger car mtr il (PCMO) technlgies cannt be used with heavy duty diesel engine il (HDEO) technlgies in the same STM. An STM must cnsist f either all PCMO technlgy r HDEO technlgy. R.5.1 Sequence IIIF If any f the Base Oils in the matrix is Grup I, the STM must cnsist f at least 7 different Base Oils and the Base Oil Prperties f Interest (R.1.2.6) include: Base Oil Saturates (ASTM D2007) Base Oil Sulfur (except when Base Oil sulfur level is less than r equal t 0.03%) (API apprved tests frm Annex E, Table E-1) Base Oil Viscsity at 100 C (ASTM D445) Base Oil Viscsity Index (ASTM D2270) Nack Vlatility f the fully frmulated il (Finished Oil) (ASTM D5800) Fr a matrix and Candidate Oil including nly Grup II, Grup III, and/r Grup IV Base Oils, the STM must cnsist f at least 5 different Base Oils and the Base Oil Prperties f Interest are: Base Oil Viscsity at 100 C (ASTM D445) Base Oil Viscsity Index (ASTM D2270) Nack Vlatility f the fully frmulated il (Finished Oil) (ASTM D5800) The relevant test results are: Percent Viscsity Increase at 80 Hurs Weighted Pistn Depsits 8

21 June 28, 2018 Phenix Meeting Average Pistn Varnish Average Camshaft plus Lifter Wear Stuck Rings The STM must cnsist f a minimum number f Base Oils cnsistent with Sectin R.2.1. Each technlgy in the STM must pass each relevant test parameter (individual passing tests r by MTEP) fr the given API categry in each Base Oil. Cnfidence intervals are applicable t each relevant test parameter except Average Camshaft plus Lifter Wear and Stuck Rings. The table shwn in R-5.1 summarizes the Base Oil Prperties f Interest that must satisfy the spread requirement (R.1.2.7). Table R-5.1 Base Oil Prperties That Must Satisfy Spread Requirement (Finished Oil) Base Oils Saturates Sulfur VI VIS100 Nack Grup I Yes N Yes N N Grup II, III, IV N N Yes N N R Detailed Example Using the Sequence IIIF with Grup I Base Oils fr API Service Categry SL Can we btain a passing Predicted Test Result that meets the requirements f the STM fr Technlgy 1 shwn in Table R-6.2 in a Candidate Oil that is within the ranges fr Base Oil saturates, sulfur, viscsity, viscsity index and finished il Nack vlatility in the IIIF? Table R-5.2 Sequence IIIF Prperties fr Example Using STM Base Oil Base Oil Saturates D 2007 Base Oil Sulfur D 4294 Finished Oil Nack Vlatility D 5800 Base Oil 100 C D 445 Base Oil Viscsity Index D 2270 IIIF Percent Viscsity Increase IIIF Weighted Pistn Depsits IIIF Average Pistn Varnish IIIF Average Cam plus Lifter Wear New Step 1: D we have enugh Base Oils in the Matrix? IIIF Stuck Rings 9

22 June 28, 2018 Phenix Meeting Yes. We have 7 Base Oils in the Matrix. The minimum number f passing Base Oils is the number f Base Oil Prperties f Interest (saturates, sulfur, viscsity at 100 C, viscsity index, and Nack vlatility f the fully frmulated il) plus tw. Step 2: D we satisfy the spread requirement fr bth saturates and Base Oil viscsity index? Yes. There are fur Base Oils with saturates belw the mean saturates f all Base Oils f 71.5 and three Base Oils abve this mean. There are tw Base Oils with a Base Oil viscsity index belw, three Base Oils abve, and tw Base Oils equal t the mean Base Oil viscsity index mean f 102. Step 3: D we pass Technlgy 1 in every Base Oil in the Matrix? Yes. Sme pass with ne test and sme pass by MTEP. (MTEP is applied t ils 1 [PVIS], 3 [WPD], and 7 [PVIS].) Step 4: D we predict a pass fr Technlgy 1 in the Candidate Oil based n the analysis f the Matrix? Yes. The predictin fr the Candidate Oil is based n a very simple mdel (see Table R-5.3), the average ver all ther Base Oils since n Base Oil effects were evident with this technlgy ver the range tested. Base Oil Base Oil Saturates D 2007 Table R Step 4: Mdel Predictin fr a New Grup I Base Oil Base Oil Sulfur D 4294 Finished Oil Nack Vlatility D 5800 Base Oil 100 C D 445 Base Oil Viscsity Index D 2270 IIIF Percent Viscsity Increase IIIF Weighted Pistn Depsits Mdel Predicted IIIF Average Pistn Varnish IIIF Average Cam plus Lifter Wear New IIIF Stuck Rings Step 5: Are there any utliers? Pssible utliers wuld include test results in which the Studentized residuals exceed the Student T distributin at the ne-sided percentile with degrees f freedm being equal t 9. t = Accrding t the calculatins in R.2.4, there are tw pssible utliers (see Table R-5.4). These utliers shuld be investigated as t their pssible cause. Given that an investigatin has nt yet taken place, the utliers are nt remved in this example. After future investigatin, the test spnsr may remve these identified utliers n a parameter-by-parameter basis. Hwever, please nte that the utlier f 2.65 identified fr Weighted Pistn Depsits CANNOT be remved unless anther test is run n this Technlgy t bring the number f Passing Base Oils in the Matrix fr Weighted Pistn Depsits back t seven. Test Number IIIF Percent Viscsity Increase Table R-5.4 Step 5: Studentized Residuals IIIF Weighted Pistn Depsits IIIF Average Pistn Varnish 10 IIIF Average Cam plus Lifter Wear IIIF Stuck Rings

23 June 28, 2018 Phenix Meeting Step 6: Is the entire 95% cnfidence interval (base d n the Student T distributin) fr the predicted mean perfrmance based n the STM mdel within the pass regin fr all relevant test prperties? If nt, is the width f the 95% cnfidence interval (based n the Student T distributin) fr the predicted mean perfrmance based n the STM mdel less than r equal t the width f the 95% cnfidence interval (based n the Nrmal Frequency Distributin and the current standard deviatin f the test used in the calculatin f severity adjustments as defined in ASTM Test Mnitring Center Technical Memrandum , Annex C, f the LTMS Manual) fr the mean based n a single test result at the predicted perfrmance level fr all relevant test prperties? Yes. Calculatins are presented belw fr Percent Viscsity Increase and summarized fr all ther test prperties. Cnfidence Interval fr the Mean Based n a Single Test Result Where: Transfrm(Result) + (Z x ) t Transfrm(Result) (Z x ) Result = Predicted Test Result fr the Candidate Oil based n the STM analysis Transfrm = Industry transfrmatin fr this test; the inverse square rt = current standard deviatin f the test used in the calculatin f severity adjustments as defined in ASTM Test Mnitring Center Technical Memrandum , Annex C, f the LTMS Manual. 1/(Result) 1/2 + (1.96 x ) t 1/(Result) 1/2 (1.96 x ) 1/(201) 1/2 + (1.96 x ) t 1/(201) 1/2 (1.96 x ) t in transfrmed units 95% cnfidence interval fr the true mean f Percent Viscsity Increase based n a single test result using the industry-published standard deviatin equals 109 t 491 The width f the cnfidence interval in riginal units equals = 382 Predicted Test Result Cnfidence Interval Width Transfrm(Result) + (t 0.05,df x S x h i ) t Transfrm(Result) (t 0.05,df x S x h i ) 11

24 June 28, 2018 Phenix Meeting Where: Result = Predicted Test Result fr the Candidate Oil based n the STM analysis Transfrm = transfrmatin used in this STM analysis: nne S = Rt Mean Squared Errr (RMSE) frm this STM analysis df = degrees f freedm used in calculating the RMSE (Result) (2.262 x x ) t (Result) + (2.262 x x ) (201) ( ) t (201) + ( ) 95% cnfidence interval fr the true mean f Percent Viscsity Increase based n the data and analysis f the STM equals 138 t 264. The width f the cnfidence interval in riginal units equals = 126. A summary f the cnfidence interval widths is shwn in Table R-5.5. Table R Summary f Cnfidence Interval Widths IIIF Parameter Cnfidence Interval Width fr a Mean Based n a Single Test Result Predicted Test Result Cnfidence Interval Width Predicted Test Result Cnfidence Interval Width Smaller? Percent Viscsity Increase YES Weighted Pistn Depsits YES Average Pistn Varnish YES Step 7: Can we btain a Predicted Test Result fr Technlgy 1 in a Candidate Oil that is within the ranges fr Base Oil saturates, sulfur, viscsity, viscsity index, and finished il Nack vlatility in the Sequence IIIF? Answer: Yes R.5.2 Sequence IIIFHD If any f the Base Oils in the matrix is Grup I, the STM must cnsist f at least 7 different Base Oils and\ the Base Oil Prperties f Interest are: Base Oil Saturates (ASTM D2007) Base Oil Sulfur (except when Base Oil sulfur level is less than r equal t 0.03%) (API apprved tests frm Annex E, Table E-1) Base Oil Viscsity at 100 C (ASTM D445) Base Oil Viscsity Index (ASTM D2270) Nack Vlatility f the fully frmulated il (Finished Oil) (ASTM D5800) Fr a matrix and Candidate Oil including nly Grup II, Grup III, and/r Grup IV Base Oils, the STM must cnsist f at least 5 different Base Oils and the Base Oil Prperties f Interest are: Base Oil Viscsity at 100 C (ASTM D445) Base Oil Viscsity Index (ASTM D2270) 12

25 June 28, 2018 Phenix Meeting Nack Vlatility f the fully frmulated il (Finished Oil) (ASTM D5800) The relevant test result is: Percent Viscsity Increase at 60 Hurs The STM must cnsist f a minimum number f Base Oils cnsistent with Sectin R.2.1. Each technlgy in the STM must pass each relevant test parameter (individual passing tests r by MTEP) in each Base Oil. Cnfidence intervals are applicable t each relevant test parameter except Ht Stuck Pistn Rings. The table shwn in R-5.6 summarizes the Base Oil Prperties f Interest that must satisfy the spread requirement (R.1.2.7). Table R-5.6 Base Oil Prperties That Must Satisfy Spread Requirement (Finished Oil) Base Oils Saturates Sulfur VI VIS100 Nack Grup I Yes N Yes N N Grup II, III, IV N N Yes N N R.5.3 Sequence IIIG If any f the Base Oils in the matrix is Grup I, the STM must cnsist f at least 7 different Base Oils and the Base Oil Prperties f Interest are: Base Oil Saturates (ASTM D2007) Base Oil Sulfur (except when Base Oil sulfur level is less than r equal t 0.03%) (API apprved tests frm Annex E, Table E-1) Base Oil Viscsity at 100 C (ASTM D445) Base Oil Viscsity Index (ASTM D2270) Nack Vlatility f the fully frmulated il (Finished Oil) (ASTM D5800) Fr a matrix and Candidate Oil including nly Grup II, Grup III, and/r Grup IV Base Oils, the STM must cnsist f at least 5 different Base Oils and the Base Oil Prperties f Interest are: Base Oil Viscsity at 100 C (ASTM D445) Base Oil Viscsity Index (ASTM D2270) Nack Vlatility f the fully frmulated il (Finished Oil) (ASTM D5800) The relevant test results are: Percent Viscsity Increase at 100 Hurs Weighted Pistn Depsits Average Cam plus Lifter Wear Ht Stuck Pistn Rings The STM must cnsist f a minimum number f Base Oils cnsistent with Sectin R.2.1. Each technlgy in the STM must pass each relevant test parameter (individual passing tests r by MTEP) in each Base Oil. Cnfidence intervals are applicable t each relevant test parameter except Ht Stuck Pistn Rings. The table shwn in R-5.7 summarizes the Base Oil Prperties f Interest that must satisfy the spread requirement (R.1.2.7). 13

26 June 28, 2018 Phenix Meeting Table R-5.7 Base Oil Prperties That Must Satisfy Spread Requirement (Finished Oil) Base Oils Saturates Sulfur VI VIS100 Nack Grup I Yes N Yes N N Grup II, III, IV N N Yes N N R Detailed Example Using the Sequence IIIG with Grup II Base Oils fr API Service Categry SN Can we btain a passing Predicted Test Result that meets the requirements f the STM fr Technlgy 1 shwn in Table R-5.8 in a Candidate Oil that is within the ranges fr Base Oil viscsity, viscsity index and finished il Nack vlatility in the IIIG? Base Oil Base Oil Saturates D 2007 Table R-5.8 Sequence IIIG Prperties fr Example Using STM Base Oil 2 Sulfur D 4294 Finished Oil Nack Vlatility D 5800 Base Oil 100 C D 445 Base Oil Viscsity Index D 2270 IIIG Percent Viscsity Increase IIIG Weighted Pistn Depsits IIIG Average Cam & Lifter Wear IIIG Number f Ht Stuck Rings Step 1: D we have enugh Base Oils in the Matrix? Yes. We have 5 Base Oils in the Matrix. The minimum number f passing Base Oils is the number f Base Oil Prperties f Interest (viscsity at 100 C, viscsity index, and Nack vlatility f the fully frmulated il) plus tw. Step 2: D we satisfy the spread requirement fr Base Oil viscsity index? Yes. There are three Base Oils with VI abve the mean f 101 and tw Base Oils belw this mean. Step 3: D we pass Technlgy 1 in every Base Oil in the Matrix? Yes. Step 4: D we predict a pass fr Technlgy 1 in the Candidate Oil based n the analysis f the Matrix? Yes. The predictin fr the Candidate Oil is based n a BOV100 mdel fr WPD, Nack fr TPVIS, and a very simple mdel fr TACLW (see Table R-5.9), the average ver all ther Base Oils since n Base Oil effects were evident with this technlgy ver the range tested. A summary f the predictin mdels are prvided belw

27 June 28, 2018 Phenix Meeting Base Oil Base Oil Saturates D 2007 Base Oil Sulfur D 4294 Table R Step 4: Mdel Predicted Finished Oil Nack Vlatility D 5800 Base Oil 100 C D 445 Base Oil Viscsity Index D 2270 IIIG Percent Viscsity Increase Mdel Predicted IIIG Weighted Pistn Depsits IIIG Average Cam & Lifter Wear NEW Step 5: Are there any utliers? Pssible utliers wuld include test results in which the Studentized residuals exceed the Student T distributin at the ne-sided percentile with the degrees f freedm being equal t 3 fr the WPD and TPVIS mdels. (The WPD and TPVIS mdels include 1 Base Oil parameter in the mdel.) The critical t value fr the TPVIS and WPD parameters is shwn belw. t = (fr WPD and TPVIS mdels) 0.05,3 TACLW predictin mdel has n Base Oil parameter in the mdel. As a result, there are 4 degrees f freedm in this mdel. This critical t value fr the TACLW parameter is shwn belw. t 0.05,4 = (fr TACLW Mdel) Accrding t the calculatins in R.2.4, there are n utliers. Test Number Table R-5.10 Step 5: Studentized Residuals IIIG Percent IIIG Weighted Viscsity Pistn Depsits Increase (TPVIS) IIIG Average Cam plus Lifter Wear (TACLW) Step 6: The 95% cnfidence intervals (based n the Student T distributin) fr the Candidate Oil are within the pass regin fr all relevant test prperties. N additinal tests fr the cnfidence interval width are required. Step 7: Can we btain a Predicted Test Result fr Technlgy 1 in a Candidate Oil that is within the ranges fr Base Oil saturates, sulfur, viscsity, viscsity index, and finished il Nack vlatility in the Sequence IIIG? Yes R.5.4 Sequence IIIGA If any f the Base Oils in the matrix is Grup I, the STM must cnsist f at least 7 different Base Oils and the Base Oil Prperties f Interest are: 15

28 June 28, 2018 Phenix Meeting Base Oil Saturates (ASTM D2007) Base Oil Sulfur (except when Base Oil sulfur level is less than r equal t 0.03%) (API apprved tests frm Annex E, Table E-1) Base Oil Viscsity at 100 C (ASTM D445) Base Oil Viscsity Index (ASTM D2270) Nack Vlatility f the fully frmulated il (Finished Oil) (ASTM D5800) Fr a matrix and Candidate Oil including nly Grup II, Grup III, and/r Grup IV Base Oils, the STM must cnsist f at least 5 different Base Oils and the Base Oil Prperties f Interest are: Base Oil Viscsity at 100 C (ASTM D445) Base Oil Viscsity Index (ASTM D2270) Nack Vlatility f the fully frmulated il (Finished Oil) (ASTM D5800) The relevant test results are: MRV TP-1 The STM must cnsist f a minimum number f Base Oils cnsistent with Sectin R.2.1. Each technlgy in the STM must pass the relevant test parameter (MTEP is nt applicable) in each Base Oil. Cnfidence intervals are nt applicable t MRV TP-1 due t the nature f test result distributin and extrardinary size f the test variability. The table shwn in R-5.11 summarizes the Base Oil Prperties f Interest that must satisfy the spread requirement (R.1.2.7). Table R-5.11 Base Oil Prperties That Must Satisfy Spread Requirement (Finished Oil) Base Oils Saturates Sulfur VI VIS100 Nack Grup I Yes N Yes N N Grup II, III, IV N N Yes N N An additinal requirement fr use f the Sequence IIIGA matrix is that the fresh il MRV f the candidate il, blended t the same viscsity grade, is equal t r less than the fresh il MRV f at least ne f the passing ils in the matrix, within the precisin f the test. ASTM D4684 MRV testing is t be carried ut at the apprpriate temperature as defined in SAE J300. R.6 Single Technlgy Matrix Examples The frm f the mdel r the mdeling technique is at the discretin f the rganizatin develping the STM. Examples f different mdeling techniques may include a simple mean with cnfidence intervals, generalized linear mdels, etc. R.6.1 Failure t Meet Spread Requirement Can we btain a passing Predicted Test Result that meets the STM requirements fr Technlgy 1 shwn in Table R in a Candidate Oil that is 75.0% saturates in a test where the pass limit is a minimum f 8.0? Fr illustrative purpses, this simplified example cnsiders Base Oil saturates as the nly prperty f interest. 16

29 June 28, 2018 Phenix Meeting Table R Test Results Technlgy Base Oil Saturates Test Result Step 1: D we have enugh Base Oils in the Matrix? Yes. We have 5 passing Base Oils in the STM which meet the minimum number f Base Oils Requirement. Step 2: D we satisfy the Spread Requirement fr saturates? N. The mean Saturates f all the Base Oils = There are fur Base Oils abve the mean and nly ne Base Oil belw the mean. Therefre, the spread requirement is nt satisfied, and we cannt analyze this Matrix as an STM. Step 3: D we pass Technlgy 1 in every Base Oil in the Matrix? N/A Step 4: Are there any Outliers? N/A Step 5: D we predict a pass fr Technlgy 1 in the Candidate Oil based n the analysis f the Matrix? N/A Step 6: Is the entire 95% Cnfidence Interval (based upn the Student T distributin) fr the predicted mean perfrmance based n the Single Technlgy Matrix mdel within the pass regin f the test r shrter than the width based n the ASTM standard deviatin? N/A Step 7: Can we btain a Predicted Test Result fr Technlgy 1 in a Candidate Oil that is within the ranges fr Base Oil saturates, sulfur, viscsity, viscsity index, and finished il Nack vlatility in the Sequence IIIF? N. The Spread Requirement has nt been met. R.6.2 Failing Base Oil Resulting in Insufficient Number f Base Oils fr the STM Can we btain a passing Predicted Test Result that meets the requirements f STM fr Technlgy 1 shwn in Table R in a Candidate Oil that is 75.0% saturates in a test where the pass limit is a minimum f 8.0? Fr this example, let s assume the nly Base Oil Prperty f Interest is Base Oil Saturates. Table R Test Results Technlgy Base Oil Saturates Test Result

30 June 28, 2018 Phenix Meeting Step 1: D we have enugh Base Oils in the Matrix? N. We have 5 Base Oils in the STM. The minimum number f passing Base Oils is five. Base Oil 2 des nt pass; Base Oil 4 is a Passing Test Result as it is an MTAC pass. Thus, with nly 4 Base Oils that pass in this technlgy, we d nt have a sufficient number f tests. Step 2: D we satisfy the Spread Requirement fr saturates and viscsity index? Yes. The mean Saturates f the 5 Base Oils = There are tw Base Oils with saturates abve the mean, tw Base Oils with saturates belw the mean, and ne equal t the mean. Therefre, the spread requirement fr saturates is satisfied. Step 3: D we pass Technlgy 1 in every Base Oil in the Matrix? N. We d nt have a pass in Base Oil 2. Step 4: Are there any Outliers? N/A Step 5: D we predict a pass fr Technlgy 1 in the Candidate Oil based n the analysis f the Matrix? N/A Step 6: Is the entire 95% Cnfidence Interval (based upn the Student T distributin) fr the predicted mean perfrmance based n the Single Technlgy Matrix mdel within the pass regin f the test r shrter than the width based n the ASTM standard deviatin? N/A Step 7: Can we btain a Predicted Test Result fr Technlgy 1 in a Candidate Oil that is within the ranges fr Base Oil saturates, sulfur, viscsity, viscsity index, and finished il Nack vlatility in the Sequence IIIF? N. There are an insufficient number f passing Base Oils in this STM. R.6.3 Mdified Technlgies Result in Insufficient Data fr STM Questin: D we have a passing Predicted Test Result that meets the requirements f STM fr Technlgy 1 shwn in Table R in a Candidate Oil that is 75.0% saturates in a test where the pass limit is a minimum f 8.0? Table R Test Results (In this hypthetical example, the number f Base Oil Prperties f Interest is Equal t 3) Technlgy Base Oil Saturates Test Result Cmments Failing Need Tech Md 18