Experimental design. Matti Hotokka Department of Physical Chemistry Åbo Akademi University

Transcription

1 Experimental design Matti Hotokka Department of Physical Chemistry Åbo Akademi University

2 Contents Elementary concepts Regression Validation Hypotesis testing ANOVA PCA, PCR, PLS Clusters, SIMCA Design of Experiments [1] Wonnacott & Wonnacott: Introductory statistics, Wiley [2] Snedecor & Cochran: Statistical Methods, Iowa State Univ. Press [3] Otto, Chemometrics, Wiley

3 Hypotesis testing Inference method Confidence levels Descriptive statistics Hypotesis testing Predictive statistics

4 Hypotesis testing Steps involved Formulate a null hypotesis This is what you want to claim E.g., the sample is within tolerances Formulate an alternative hypotesis This is a complement to null hypotesis E.g., the sample is not within tolerances Calculate a characteristic number Compare with tabulated values Accept or reject the null hypotesis

5 Hypotesis testing Huge number of test exist Tests for mean Tests for distribution Tests for spread Tests for outliers Etc.

6 Hypotesis testing Test for the mean Double-sided t-test, x = x µ t = s P(X) n Acceptable No-no No-no x

7 Hypotesis testing Mean at a nominal value The ibuprofen concentration must be 400 mg per pill. Therefore 400 mg. Take 5 pills and measure the ibuprofen content. The results are 396, 388, 398, 382, 373 mg. Mean x = 387 mg, s = 10.3 mg. Calculate the critical number, t = 2.82 Degrees of freedom = n-1 = 4 Choose risk level: 5 % (95 % confidence) Read the table for Student s t-test at risk level because the risk 2.5 % at the low end and 2.5 % at the high end gives total risk of 5 %. The value in the table, 2.776, is smaller than the calculated one. Reject the null hypotesis. Accept the alternative hypotesis. We cannot guarantee at 95 % confidence level that the pills have the precribed amount of ibuprofen. =

8 Student s distribution D.f. Risk N = number of samples D.f. = degrees of freedom = N - 1 This table is one-sided. Therefore the total risk at level is 2.5 % % and confidence probability is 95 %.

9 Hypotesis testing Test for the mean One-sided t-test, x = x t = µ s P(X) n Acceptable No-no x

10 Hypotesis testing Mean below a nominal value The EU regulatory limit for nitrate in drinking water is 50 mg/l. Determinations from 4 parallel samples gave the results 51.0, 51.3, 51.6, 50.9 mg/l. Is this just random variation or is the observed level systematically above the prescribed limit? Mean 51.2 and st.dev mg/l. Null hypotesis: the level is not exceeded, x it is too high. Calculate t = Choose risk level: 5 %. D.f. = 4-1 = 3., alternative hypotesis: The tabulated value of t, 2.353, is smaller than the calculated one. The null hypotesis must be rejected. The concentration is too high.

11 Hypotesis testing Compare two means Compare two sets of parallel measurements from different samples. Do the two samples differ significantly? A two-sided test. t = x s d x n n n + n s d = 2 ( n1 1) s1 + ( n 1) s n + n D. f.= n1 + n2 2

12 Hypotesis testing Do two production batches differ? Quality control tests the day and night shifts at a refinery. The octan numbers of parallel measurements are (1: day) 94.92, 95.07, 94.96, 95.02, 94.99, 94.93; (2: nite) 95.03, 95.08, 94.98, 95.03, 95.01, Means: (1) 94.98; (2) St.dev.: (1) 0.057; (2) Weighted st.dev. = Student s t = d.f. = 10 Choose risk level 2.5 %, read column : t = Comparison: No, we cannot say that the two results differ. Therefore only random variations are observed.

13 Q 1 Hypotesis testing = Dixon s Q test for outliers Can be applied also for very few observations. Arrange your n observations in ascending order. Calculate the numbers Q 1 and Q n. Null hypotesis: not an outlier. Accepted if calculated Q less than tabulated. x x n x x ; Q 2 = xn x n xn x 1 1

14 Hypotesis testing Dixon s Q test for outliers Critical values of Q test at the 1 % risk level. Number of observations = n. n Q n Q

15 Hypotesis testing Dixon s test for outliers Personer i följande åldrar deltar i en bussresa till teater i Helsingfors: 6, 7, 5, 6, 7, 6, 103, 8, 7, 5. Order them: 5, 5, 6, 6, 6, 7, 7, 7, 8, 103. Q 1 = 0, 5 is not an outlier; Q 2 = 0.969, 103 certainly is an outlier.

16 Hypotesis testing Grubb s test for outliers Observation x* is not an outlier in a series if T = x s x * < T Tabulated

17 Hypotesis testing Grubb s outlier test Critical values for Grubb s outlier test at 95 % and 99 % levels. Number of observ ations = n. n T(95%) T(99%) n T(95%) T(99%)

18 Hypotesis testing Outliers in linear regression In order to find whether or not observation k (value y k ) is an outlier 1) Calculate a new regression with observation k removed. 2) Calculate e k = y k obs - y k calc.

19

20 ANOVA Analysis of variance Used to test interdependences between batches. Used as an analysis tool for designed experiments. Requires several parallel measurements (replicates) of each batch (or experiment).

21 Anova One-way analysis Assume that four different samples are taken from waste water of a factory to study the potassium concentration (mg/l). Each sample is analysed by a different crues. Three parallel measurements are made to determine the concentration of each sample. Replicate Batch Mean

22 Anova Variation between samples Replicate Batch Mean y total = SSQ fact = 0.489

23 Anova Variation between samples Replicate Batch Mean y total = SSQ fact = SSQ = n ( y y ) 2 fact j j total j= 1

24 Anova Variation within samples Replicate Batch Mean y total = SSQ fact = SSQ R = (y ij - y j) 2

25 Anova Variation within samples Replicate Batch Mean y total = SSQ fact = (y ij - y j) 2 SSQ R = n SSQ = ( y y ) 2 j R ij j j= 1 i= 1

26 Anova Total variation Replicate Batch Mean y total = SSQ fact = SSQ R = SSQ corr = 0.749

27 Anova Total variation Replicate Batch Mean y total = SSQ fact = SSQ R = SSQ corr = q n SSQ = ( y y ) 2 j corr ij total j= 1 i= 1

28 Anova Total variation broken down to contributions Replicate Batch Mean y total = SSQ fact = SSQ R = SSQ corr = SSQ corr = SSQ fact + SSQ R

29

30 PCA Principal component analysis PCA finds a direction along which the points lie. X = Ca ph Features Object!!!

31 PCA What does it mean? y=ph Principal component (1 1) x=ca

32 PCA What is it? PCA classifies the observations. It does not perform any regression. X = 2 " $ $ $ # Low % ' ' ' & Medium High

33 PCA What does it mean? y=ph Principal component (1 1) x=ca

34 PCA Next principal component The next principal direction with the next largest spread must be orthogonal to the first one.

35 PCA The second principal component y=ph Principal component (1-1) Principal component (1 1) x=ca

36 ( PCA How is it done? Direction of largest spread needs to be found. Spread along the coordinate axes is given by the variance-covariance matrix. Its eigenvalue gives the characteristic spread. The corresponding eigenvector gives the direction. Eigenvectors are automatically orthogonal. So, diagonalize the Only, you don t. matrix.

37 PCA How is it done? Diagonalization gives ALL eigenvalues. You only need a few largest. Use special mathematical techniques instead.

38 PCA Eigenvalues The spread of the first component is largest, that of the second smaller etc. Two or three components usually explain all the spread down to experimental errors. Eigenvalue = Spread These do not differentiate the observations Component

39 PCA How is it done, then? Break down the observations X to a product of a scores matrix T and a loadings matrix L. X = 2 ) * ,... - T = 2 / Scores L T = ( 1 1) Loadings X = T L T Compare: y = ax This example is mathematically inconsistent!

40 PCA Loadings The loadings matrix tells what is the direction of the principal component. y=ph Principal component, L T = (1 1) x=ca

41 PCA Scores The scores matrix tells where the points lie along the new coordinate axis. y=ph x=ca

42 PCA A real case Hairs samples from a crime site were analyzed. The following elemental compositions of the hairs of the suspects were detected. Hair Cu Mn Cl Br I

43 PCA Scores Consider two principal components: PC PC1

44 PCA Loadings Loadings tell how much the original variables contribute to the principal component: PC2 I Mn 0 Cu Mn Br I Cl Cu Cl 0 PC1 0 Br PC1

45 PCR Principal component regression (Multivariate) linear regression along the principal components. Only one (or a few) variable(s). Maximal resolving power.

46 5 PLS Partial least squares Linear regression: y = x a Multivariate regression: y = X a y = X a PLS: y = U Q y = U Q

47

48 Cluster analysis Cluster analysis finds observations that are more similar to each other than to observations outside the cluster.

49 Cluster analysis Distance Cluster analysis is based on distance (or similarity) between objects. City-block distance Euklidian distance Pearson-distance Mahalonobis distance...

50 Cluster analysis City-block distance Feature 2 x 22 d 12 = x 11 -x 21 + x 12 -x 22 x 12 -x 22 x 12 x 11 -x 21 x 11 x 21 Feature 1

51 Cluster analysis Euklidian distance Feature 2 x 22 d 12 = [(x 11 -x 21 ) 2 + (x 12 -x 22 ) 2 ] 1/2 x 12 x 11 x 21 Feature 1

52 Cluster analysis Pearson-distance d ij = K k = 1 ( x x ) ik s 2 j jk 2

53 Cluster analysis Example data Concentrations of calcium and phosphate in six blood serum samples (mg per 100 ml). Object Features Calcium Phosphate d 12 = [( ) 2 + ( ) 2 ] 1/2 = 0.354

54 Cluster analysis Distance matrix Object Smallest distance *

55 Cluster analysis Second distance matrix Object 1* * * 4*

56 Cluster analysis Third distance matrix Object 1* 3 4* 5 1* * * 4* 5*

57 Cluster analysis Fourth distance matrix Object 1* 3 5* 1* * * 1* 4* 5*