BASICS OF CHEMOMETRICS

Transcription

1 INTRODUCTION - Chemometrics Introduction What is this and why we need it BASICS OF CHEMOMETRICS Juan Antonio Fernández Pierna Vincent Baeten Pierre Daenne - Some definitions - Overview of methods - Eamples Walloon Agricultural Research Centre (CRA-W) Valorisation of Agricultural Products Department Gemblou, Belgium 2 X - METRICS CHEMOMETRICS INTRODUCTION The use of multivariate analysis in the discipline X: Statistical, mathematical or graphical technique, considers multiple variables simultaneously Chemometrics is the chemical discipline that uses mathematics and statistics to design or select optimal eperimental procedures, to provide maimum relevant chemical information by analyzing chemical data, and to obtain knowledge about chemical systems - Biometrics (used in biology) - Technometrics (used in engineering) - Psychrometrics (used in phychology) - Chemometrics (used in chemistry) 3 D. L. Massart 4

2 CHEMOMETRICS INTRODUCTION CHEMOMETRICS INTRODUCTION Mathematics Computing Engineering Statistics Theoretical and physical chemistry Biology Analytical chemistry Organic chemistry Industrial Food Feed The scientific world today The data flood generated by modern analytical instrumentation produces large quantity of numbers to understand and quantify phenomenons around us. The evolution of personal computers allows faster acquisition, processing and interpretation of chemical data. Every scientist uses software related to mathematical methods or to processing of knowledge. Meeting point of various disciplines Chemometrics 5 A deeper understanding of those methods and tools for viewing all data simultaneously are needed. 6 CHEMOMETRICS INTRODUCTION CHEMOMETRICS INTRODUCTION Use of mathematical and statistical methods for selecting optimal eperiments Statistical eperimental design Design of Eperiments (DoE) CHEMOMETRICS S Analytical request Analytical method Analytical answer Etracting maimum amount of information when analysing multivariate (chemical) data Classification Process monitoring, Multivariate calibration Useful at any point in an analysis, from the first conception of an eperiment until the data is discaed. 7 8

3 Wavelength Wavelength Wavelength CHEMOMETRICS APPLICATIONS CHEMOMETRICS DEFINITIONS Huge growth area in past 5 years Process Control and analysis Food and feed analysis Biology metabolomics etc Environmental monitoring Analytical Chemistry Linear algebra is the language of Chemometrics. One cannot epect to truly understand most chemometric techniques without a basic understanding of linear algebra (Wise and Gallagher, 998) Matri and vector operations 9 CHEMOMETRICS DEFINITIONS CHEMOMETRICS DEFINITIONS - Samples are referred to as OBJECTS - Measurement results (e.g. concentration, absorbances, ) are referred to as VARIABLES -A data table of K variables and M objects is referred to as a DATA MATRIX OF SIZE MK M objects - observations X K variables CHEMOMETRICS: Etract meaningful information about the objects and the variables from data matrices é ê ê 2 X = ê... ê êë n n2 Variables... pù... ú 2 pú ú ú... npú û Samples If X has 3 rows and 5 columns 3 5 data matri Absorbance Absorbance Absorbance 2

4 CHEMOMETRICS INTRODUCTION CHEMOMETRICS IMPORTANT DISCIPLINES In summary M objects X K variables = é ê ê 2 ê... ê êë n n pù ú 2 pú... ú ú npúû Sampling, selection of objects and variables Clustering Multivariate regressions, calibrations and predictions Neural Networ Validation Graphical display and outlier detection 3 4 Source: Chemometrics Introduction - Jens C. Frisvad Calibration data set (X) Sample Selection SAMPLE SELECTION Data Eploration Pattern Recognition Visualization Unsupervised analysis Principal Component Analysis (PCA) Data matri pre-treatment Supervised analysis Regression models Model building Model construction Calibration INCLUDE IN THE CALIBRATION SET ALL THE FACTORS OF VARIATION: Measurement factors: - room temperature - operator - instrument setup clustering tendency outlier detection Outliers Validation New data Prediction Selection optimal compleity Uncertainty estimation 5 Population sources of variation: - origins of samples - processes - varieties - storage conditions - sample preparations (t, particle size) - residual moisture 6

5 PATTERN RECOGNITION PRINCIPAL COMPONENT ANALYSIS Groupingofobjectse.g.how similaristhebehaviourofcompounds, how similar are products PCA PrincipalComponentAnalysis PCA isachemometrictechniqueforvisualizing high dimensionaldata.pca reducesthedimensionality(the numberofvariables)ofadatasetbymaintainingasmuch varianceaspossible PC Lookingatrelationships: Betweensamples:food samples / patients / people / spectra / PC 2 (3.42%) PC2 Betweenvariables:elementalcompositions/compound concentrations / spectral pea / PC2.6 PC PC (96.9%) PRINCIPAL COMPONENT ANALYSIS CALIBRATION.2 Samples/Scores Plot of Xoffset Quantitativeorqualitativeestimation.Especialymitures. Scores on PC 3 (.34%) Clusters Outliers Univariatecalibration:onemeasuremente.g.apeakhight Multivariatecalibration:severalmeasurementse.g.spectra Statistical,mathematicalorgraphicaltechnique,considers multiplevariablessimultaneously Scores on PC (96.9%) Object space 9 2

6 MULTIVARIATE CALIBRATION MULTIVARIATE CALIBRATION Relateinstrumentalresponsetochemicalconcentration. Impropercalibrationyieldsimproperresults. Linear,non-linearandmultivariatecalibrationalgorithms areavailable,dependingontheinstrumentandtheanalysis involved. Linking two sets of data together : peak hight to concentration /Spectra to concentrations /biological activitytostructure/ 2 Calibration X Data spectra Y Data + Referenceues Calibration model -Moisture(regression) -Concentrationofprotein(regression) -countryoforigin(discrimination) -PDO (discrimination) - -Infraredspectroscopy(IR-Raman) -Fourier-Transform Infraredspectroscopy(FTIR) -NuclearMagneticResonance(NMR) -X-RayFluorescence(XRF) -X-RayDiffraction(XRD)spectroscopy - 22 MULTIVARIATE CALIBRATION - VALIDATION MULTIVARIATE CALIBRATION Calibration X Data spectra + Y Data Refue Calibration model Y=f(X)=XB+E Validation X Data spectra + Calibration model Y Data /refue. Y is a matrinm containingthe reference ues X is a matrinp containingthe spectra (NIR,MIR, Raman, ) B is a matripm containingthe regression coeficients Calibration uses empiricaldata and prior knowledge for determininghow to predict unknown quantitative information yfrom available measurements,via some mathematicaltransfer functions 23 E is a matrinm eplainingfor the modelerror. 24

7 Unsupervised Principal Component Analysis (PCA) Cluster Analysis (CA) Multiple Linear Regression (MLR) Unsupervised - Pattern recognition Multivariate analysis Regression Principal Component Regression (PCR) Partial Least Squares (PLS) ArtificialNeural Networ (ANN) Unsupervised Principal Component Analysis (PCA) Cluster Analysis (CA) Multiple Linear Regression (MLR) Principal Component Regression (PCR) Supervised LS Support Vector Machines (LS-SVM) Local techniques Multivariateanalysis Regression Partial Least Squares (PLS) Artificial Neural Networ (ANN) LS Support Vector Machines (LS-SVM) Discrimination PLS-Discriminant Analysis (PLS-DA) SIMCA Supervised Local techniques à See similarities and regularities present in the data. PLS-Discriminant Analysis (PLS-DA) Discrimination SIMCA K-Nearest Neighbours (k-nn) K-Nearest Neighbours (k-nn) 25 Support Vector Machines (SVM) 26 Algorithms Support VectorMachines (SVM) Supervised - Regression Supervised - Discrimination Unsupervised Principal Component Analysis (PCA) Cluster Analysis (CA) Multiple Linear Regression (MLR) Principal Component Regression (PCR) Principal Component Analysis (PCA) à Regression Unsupervised based statisticaltechnique used in determining which Cluster Analysis particular classification or Multiple Linear Regression (MLR) (CA) group an item of data or an object belongs to on the basis of its characteristics or essentialfeatures. Principal Component Regression (PCR) Partial Least Squares (PLS) Partial Least Squares (PLS) Multivariateanalysis Regression Artificial Neural Networ (ANN) Multivariateanalysis Regression Artificial Neural Networ (ANN) LS Support Vector Machines (LS-SVM) LS Support Vector Machines (LS-SVM) Supervised Local techniques Supervised Local techniques PLS-Discriminant Analysis (PLS-DA) PLS-Discriminant Analysis (PLS-DA) à It includes any Discrimination techniques for modeling SIMCA and analyzing several variables,when the focus is on the K-Nearest relationship Neighbours (k-nn) between a dependent variable (property of interest) and one or more independent Support Vector Machines (SVM) variables (spectra). 27 Discrimination SIMCA K-Nearest Neighbours (k-nn) Support Vector Machines (SVM) 28

8 VARIABLE SELECTION VARIABLE SELECTION Find a setofpredictorvariableswhich givesagoodfit, predicts the dependent ue wel and is as smal as possible. Backwaelimination:Startwithalthepredictorsand potentialydroppredictorsinsubsequentsteps. Forwaselection:Startwithnopredictorsandpotentialy addpredictorsinsubsequentsteps. Stepwiseregression:Combinationofbackwa eliminationandforwaselection VARIABLE SELECTION CHEMOMETRICS SOFTWARE Reducingnumberofvariables Possibilityofconstructingfastspectrometersbasedonareduced number of variables aiming to a reduction of ing and utilizationtimestobeused,forinstance,inaconveyerbelt. Moreover,thefactofgroupingthedifferentselectedvariablesin regionsalowsaneasychemicalinterpretationofthespectra. References - Backwavariableselectionin PLS (BVSPLS) ;J.A.Fernández Pierna,V.Baeten,P.Daenne.AnalyticaChimicaActa642(29) pp Prediction errorimprovementsusing variableselection on smal calibration sets - a comparison of some recentmethods.s.i. Overgaa,J.A.Fernández Pierna,V.Baeten,P.Daenne & T. Isason.J.NearInfraredSpectrosc.2, (22). 3 32

9 EXAMPLE: FEED EXAMPLE: FEED Other FARIMAL Vegetal Other Poultry Bovine + Pig Terrestrialanimal Fish Belonging to. CAL LOOCV % classified as. % classified as. Fish Rest Fish Rest Fish Rest %correct classification EXAMPLE: CEREALS - IMPURITIES EXAMPLE: NUTS AND DRIED FRUITS Walnut Hazelnut Brazil nut Almond Cashew Raisin SVM discrimination models NIR hyperspectral imaging spectroscopy and chemometrics for the detection of undesirable substances in food and feed J.A. Fernández Pierna, Ph. Vermeulen, O. Amand, A. Tossens, P. Daenne and V. Baeten. Special issue Chemometrics and Intelligent Laboratory Systems 7 (22)

10 EXAMPLE: OLIVES - IMPURITIES EXAMPLE: OLIVE OIL QUALITY PARAMETERS KNN - PCA models Usinga visible vision system for on-line determination of qualityparameters of olive fruits. E. Guzmán, V.Baeten,J.A.FernándezPierna,J.A.García Mesa.Foofand Nutrition Sciences,4,9-98(23) Application of low-resolution Raman spectroscopyfor the analysis of oidized olive oil E. Guzmán, V.Baeten,J.A.FernándezPierna,J.A.García Mesa.(2) Food control 22, EXAMPLE: OLIVE OIL ADULTERATION EXAMPLE: SUGAR BEET PLANTS CYST optical microscopy NIR hyperspectral imaging SVM discrimination models Detection ofthe Presence ofhazelnut Oil in Olive Oil by FT-Raman and FT-MIR Spectroscopy'V.Baeten, J.A.FernándezPierna,P.Daenne,M.Meurens,D.L.García González,R.Aparicio.Journal of Agricultural and Food Chemistry,53 (6) (25), NIRhyperspectral imagingspectroscopyand chemometrics for the detection of undesirable substances in food and feed J.A. Fernández Pierna, Ph. Vermeulen, O. Amand, A. Tossens, P. Daenne and V. Baeten. Special issue Chemometrics and Intelligent Laboratory Systems 7 (22)

11 EXAMPLE: SUGAR BEET PLANTS CERCOSPORA EXAMPLE: FEED PRODUCTS Multivariate regression method comparison: PLS, ANN and LS-SVM 4 42 EXAMPLE: FEED PRODUCTS EXAMPLE: FEED PRODUCTS Feed (286767) Ash, Fat, Fibre, Starch, Protein Feed Ingredients (266527) Ash, Fat, Fibre, Protein Fresh Silages (357) Dry Matter, Fibre, Protein Soils (6257) CEC, COT_SK, N_Kj Pre-processing: SNV + detrend + First derivative 43 Eample of data distribution (Soils N_Kj) 44

12 EXAMPLE: FEED PRODUCTS EXAMPLE: FEED PRODUCTS Feed - Fat Feed - fibre Feed - ash Feed ingredients - ash Feed ingredients - fat 2.5 Feed - starch Feed - protein Feed ingredients - fibre Feed ingredients - protein EXAMPLE: FEED PRODUCTS EXAMPLE: TRANSFERT CALIBRATION TRANSFER FROM DISPERSIVE INSTRUMENTS TO HANDHELD SPECTROMETERS (MEMS) Fresh silages - fibre Fresh silages - protein Fresh silages - dry matter Calibration Transfer from Dispersive Instruments to Handheld Spectrometers, J.A. Fernández Pierna, P. Vermeulen, B. Lecler, V. Baeten, P. Daenne. Applied Spectroscopy 64 (6) (2) 48

13 EXAMPLE: TRANSFERT QUESTIONS ABOUT CHEMOMETRICS? j.f cra.walonie.be http: /