Outline. The ordinal package: Analyzing ordinal data. Ordinal scales are commonly used (attribute rating or liking) Ordinal data the wine data

Transcription

1 Outline Outline The ordinal package: Analyzing ordinal data Per Bruun Brockhoff DTU Compute Section for Statistics Technical University of Denmark August c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 1 Income group data Soup data (Paired) Degree-of-difference data -Alternative choice Cumulative link models 3 4 Cumulative Link Mixed Models 5 Thurstonian -AC model via CLMMs c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics 015 / 34 Ordinal scales are commonly used (attribute rating or liking) Ordinal data the wine data 5-point scales: 7-point scales: Randall, J (1989). The analysis of sensory data by generalised linear model. Biometrical journal 7, AND: included in ordinal pacakge. A sensory experiment: Temperature and contact between juice and skins can be controlled when crushing grapes during wine production. These factors are thought to affect the bitterness of the wine. 9-point scales: Objective: How does perceived bitterness depend on temperature and contact? c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34

2 Ordinal data the wine data Data for the bitterness of white wines Objective: How does perceived bitternes depend on temperature and contact? Table: (Randall, 1989), N=7 Variables Type Values bitterness response 1,, 3,4, 5 less more temperature predictor cold, warm contact predictor no, yes judges random 1,..., 9 Temperature and contact between juice and skins can be controlled when cruching grapes during wine production. Table: Ratings of the bitterness of some white wines. Data are adopted from Randall (1989). Judge Temperature Contact Bottle cold no cold no cold yes cold yes warm no warm no warm yes warm yes Bitterness ratings: 1(least),, 3, 4, 5(most) c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 Income group data Soup data Ordinal data Income group data McCullagh, P. (1980) Regression Models for Ordinal Data. Journal of the Royal Statistical Society. Series B (Methodological), Vol. 4, No.., pp AND: included in ordinal pacakge. head(income) year pct income c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 Ordinal data Soup data Christensen, R. H. B., Cleaver, G., & Brockhoff, P. B. (011). Statistical and Thurstonian models for the A-not A protocol with and without sureness. Food Quality and Preference,, Industrial product development experiment - Unilever. AND: included in ordinal pacakge. A-not A with sureness scale: head(soup) Reference Not Reference Sure Not Sure Guess Guess Not Sure Sure RESP PROD PRODID SURENESS DAY SOUPTYPE SOUPFREQ COLD EASY GENDER 1 1 Ref Canned >1/week Yes 7 Female 1 Test 5 1 Canned >1/week Yes 7 Female 3 1 Ref Canned >1/week Yes 7 Female 4 1 Test Canned >1/week Yes 7 Female 5 1 Ref 1 5 Canned >1/week Yes 7 Female 6 1 Test 6 5 Canned >1/week Yes 7 Female AGEGROUP LOCATION Region Region 1 c 3 Per Bruun Region Brockhoff 1 (DTU) Region 1 The ordinal package: Analyzing ordinal data DTU Sensometrics / Region 1

3 (Paired) Degree-of-difference data -Alternative choice Ordinal data (Paired) Degree-of-difference data Ordinal data The -Alternative choice test (-AC) 5 panelists 8 replications. Table: Paired degree-of-difference test, data adopted from (?) Pair Identical Uncertain Different Total Identical pair Different pair Christensen, R. H. B. and P. B. Brockhoff (011) Analysis of replicated categorical ratings data from sensory experiments. Journal of the French Statistical Society, SFdS, 154(3), Alternative Choice (-AC): Do you prefer A or B or do you not have a preference? Which is strongest, A or B, or is there no difference? Table: 08 consumers with 4 replications Condition Prefer A No-preference Prefer B Total A B Christensen, R. H. B., H.-S. Lee and P. B. Brockhoff (01) Estimation of the Thurstonian model for the -AC protocol. Food Quality and Preference, 4, c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 -Alternative choice Cumulative link models Appropriate models for ordinal data Understanding the cumulative link model Y: Ordinal data not continuous data A linear regression model on the scores (1,...,5)? Breach of assumptions: The scores are not normally distributed A score of 4 is not twice as much as Variance not likely to be constant Our approach: A cumulative link model (CLM) Only use information about ordering Intuitively: A linear model that respects the ordinal nature of the response β P(Y = cold) θ 1 α θ θ 3 θ 4 warm cold Latent bitterness follows a linear model: S i = α + x i β + ε i, ε i N(0, σ ) = α + β(temp i ) + ε i We only observe a grouped version of S i : θ j 1 S i < θ j Y = j P (Y i j) = F (θ j x i β) c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34

4 Cumulative link models Fitting cumulative link models with clm data(wine) fm1 <- clm(rating ~ contact + temp, data=wine, link="probit") summary(fm1) formula: rating ~ contact + temp data: wine link threshold nobs loglik AIC niter max.grad cond.h probit flexible (0) 1.43e-13.e+01 Coefficients: Estimate Std. Error z value Pr(> z ) contactyes ** tempwarm e-07 *** --- Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Threshold coefficients: Estimate Std. Error z value c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 Cumulative link models A cumulative link model for the wine data Additive effects for temperature and contact: P (Y i j) = Φ (θ j β 1 (temp i ) β (contact i )) Is there an interaction between temp and contact? Table: ANODE table for the wine data. Source df deviance p value Total < Treatment < Temperature, T < Contact, C < Interaction, T C Residual c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 An extended CLM Framework Understanding structured thresholds restrictions Standard CLM: F (θ j x i β) Y: β Extended CLM: ( g(θj ) wi F β j x i β ) Threshold effects Nominal effects exp(zi ζ) Scale effects CLMM (Mixed effects): F (θ j fixed Xβ random Zb ) P(Y = cold) warm cold The cumulative link model: P (Y i j) = F (θ j β(temp i )) θ j ordered, but otherwise not restricted Require symmetry? Require equidistance? θ 1 α θ θ 3 θ 4 c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34

5 Fitting models with structured thresholds Sensory applications of structured thresholds fm.equi <- clm(rating ~ contact + temp, data=wine, link="probit", threshold="equidistant") summary(fm.equi) formula: rating ~ contact + temp data: wine link threshold nobs loglik AIC niter max.grad cond.h probit equidistant (0) 1.40e-08 3.e+01 Coefficients: Estimate Std. Error z value Pr(> z ) contactyes ** tempwarm e-07 *** --- Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Threshold coefficients: Estimate Std. Error z value threshold spacing Symmetric thresholds for sureness scales: Reference Not Reference Sure Not Sure Guess Guess Not Sure Sure Equidistant thresholds for 7- or 9-point preference scales: Equally spaced categories is a necessary condition for using linear models for continuous data on ordinal data. c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 Understanding scale effects in CLMs Thurstonian model for the A-not A with sureness protocol Y: β Model for latent bitterness: S i = α + β 1 (temp i ) + β (contact i ) + ε i, Reference Test products products N(0, 1) N(δ, σ ) δ warm cold ε i N(0, σ (temp i )) Mccullagh, 1980, Cox, 1995, Agresti 00 ( ) θj β 1(temp γ ij = F i ) β (contact i) ζ 1(temp i ) θ 1 θ θ 3 θ 4 θ 5 Sensory intensity Reference Not Reference Product Sure Not Sure Guess Guess Not Sure Sure Reference Test θ 1 α θ θ 3 θ 4 Table: Discrimination of packet soup Christensen, Cleaver and Brockhoff, 011 c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34

6 Cumulative Link Mixed Models Including random effects in CLMs Cumulative Link Mixed Models Cumulative link mixed models β warm The cumulative link model: γ ij = F (θ j β 1 (temp i ) β (contact i )) Judges perceive wine bitterness differently Judges use the response scale differently Add random effects for judges: γ k = F (B k ψ Zv o k ) V N(0, Σ τ ) The log-likelihood function: l(ψ, τ ; y) = log p ψ (y v)p τ (v) dv R r Integration methods: Laplace approximation Tierney and Kadane, 1986, Pinheiro and Bates, 1995, Joe 008 cold γ ij = F (θ j β 1 (temp i ) β (contact i ) b(judge i )), b N(0, σ b ) Gauss-Hermite quadrature (GHQ) Hedeker and Gibbons, 1994 Adaptive Gauss-Hermite quadrature (AGQ) Liu and Pierce, 1994 θ 1 α θ θ 3 θ 4 A Newton-Raphson algorithm updates the conditional modes of the random effects (Laplace and AGQ) c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics 015 / 34 Allowing for differences between judges ANODE for mixed effects CLM Research questions: Are judges rating the wines differently? Are there differences between bottles? Additive random effects for judges: P (Y i j) = Φ (θ j β 1 (temp i ) β (contact i ) u(judge i )) u(judge i ) N(0, σu) Additive random effects for judges and bottles: P (Y i j) = Φ (θ j β 1 (temp i ) β (contact i ) u(judge i ) b(bottle i )) u(judge i ) N(0, σu) b(bottle i ) N(0, σb ) Results: Table: ANODE table for the wine data with random effects. Source df deviance p value Total < Var(Judge) < Var(Bottle) Treatment < Temperature, T < Contact, C < Interaction, T C Bottles are probably not that different Judges do rate the wines differently c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34

7 Panel inference judge effects 5 panelists 8 replications. Judge effect Table: Paired degree-of-difference test, data adopted from (?) Pair Identical Uncertain Different Total Identical pair Different pair Judge Christensen, R. H. B. and P. B. Brockhoff (011) Analysis of replicated categorical ratings data from sensory experiments. Journal of the French Statistical Society, SFdS, 154(3), c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 Thurstonian model for the A-not A with sureness protocol 5 panelists 8 replications. Table: Comparison of tests of product differences. Test χ -value df p-value Naive Pearson test Stuart-Maxwell test (?) LR test in CLMM Reference Test products products N(0, 1) N(δ, σ ) δ θ 1 θ θ 3 θ 4 θ 5 Sensory intensity Christensen, R. H. B. and P. B. Brockhoff (011) Analysis of replicated categorical ratings data from sensory experiments. Journal of the French Statistical Society, SFdS, 154(3), c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 Reference Not Reference Product Sure Not Sure Guess Guess Not Sure Sure Reference Test Table: Discrimination of packet soup Christensen, Cleaver nd Brockhoff, 011 c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34

8 Including assessor effects Inference for respondents respondent-specific d s Assumptions: Assessors do not use the response scale differently 3 Assessors do not have different d s Accommodate this with mixed model extensions: Allow normally distributed random effects for assessors P (S i θ j ) = Φ (θ j δ(prod i ) u(assessor i )) u N(0, σu) d prime (confidence interval) 1 0 Note: This is similar to assessor effects in models for sensory profiling! c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 Assessors c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 Thurstonian -AC model via CLMMs Thurstonian -AC model via CLMMs The -Alternative choice test (-AC) Thurstonian model for the -AC protocol -Alternative Choice (-AC): θ 1 θ Do you prefer A or B or do you not have a preference? Which is strongest, A or B, or is there no difference? B A ~ N(d', ) B A d' ~ N(0, 1) π 1 π π 3 π 1 π π 3 Table: 08 consumers with 4 replications τ 0 τ d' A stronger than B no difference B stronger than A τ d' 0 τ d' Condition Prefer A No-preference Prefer B Total A B Christensen, R. H. B., H.-S. Lee and P. B. Brockhoff (01) Estimation of the Thurstonian model for the -AC protocol. Food Quality and Preference, 4, c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 The Thurstonian model for the -AC protocol can be formulated as a cumulative link model: ˆτ = (ˆθ ˆθ 1 )/ ˆδ = ( ˆθ ˆθ 1 )/ se(ˆτ) = {var(θ ) + var(θ 1 ) cov(θ, θ 1 )}/ se(ˆδ) = {var(θ ) + var(θ 1 ) + cov(θ, θ 1 )}/ c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34

9 Thurstonian -AC model via CLMMs Outline Illustrating the model Outline Average consumer 5th percentile consumer Reference A Reference B 95th percentile consumer prefer A no preference prefer B 95% of population within ±1.96σ d = ±3.3 (d units) The largest effect is consumer differences: χ 1 = 153.6, p < Effect of reference in duo-trio test only for consumers with an average preference 1 Income group data Soup data (Paired) Degree-of-difference data -Alternative choice Cumulative link models 3 4 Cumulative Link Mixed Models 5 Thurstonian -AC model via CLMMs c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34 Excercises - Day afternoon Excercises - Day afternoon NO ordinal exercises IF you want: look at ordinal vignettes and/or manuel/help material Recommend instead - work with exercises from the previous three teaching blocks: sensr part 1 exercises sensr part exercises lmertest exercises SensMixed exercises c Per Bruun Brockhoff (DTU) The ordinal package: Analyzing ordinal data DTU Sensometrics / 34