Advances in Sensory Discrimination Testing: Thurstonian-Derived Models, Covariates, and Consumer Relevance

Transcription

1 Advances in Sensory Discrimination Testing: Thurstonian-Derived Models, Covariates, and Consumer Relevance John C. Castura, Sara K. King, C. J. Findlay Compusense Inc. Guelph, ON, Canada

2 Derivation of a psychometric function

3 A

4 A B

5 A δ B

6 A δ B dʹ is an estimate of δ

7 A δ B dʹ=1 corresponds to 76% correct on 2AFC

8 Psychometric function p c = f psy (dʹ)

9 Psychometric function p c = f psy (dʹ)

10 Psychometric function p c = f psy (dʹ)

11 Psychometric function p c = f psy (dʹ) f psy encodes psychological decision-making rules into a mathematical model, and is used to create a mapping between p c and dʹ

12 A B A 1 A 2 B

13 Psychometric function: 3AFC A 1 A 2 B

14 Psychometric function: 3AFC A 1 < B A 1 A 2 B

15 Psychometric function: 3AFC A 1 < B A 2 < B A 1 A 2 B

16 Psychometric function: 3AFC A 1 < B A 2 < B A 1 A 2 B

17 Psychometric function: 3AFC A 1 < B AND A 2 < B A 1 A 2 B

18 Psychometric function: 3AFC A 1 < B AND A 2 < B A 1 A 2 B

19 Psychometric function: 3AFC A 1 < B A 2 < B A 1 A 2 B

20 Psychometric function: 3AFC A 1 < B B < A 2 A 1 B A 2

21 Psychometric function: 3AFC A 1 < B B < A 2 A 1 B A 2

22 Psychometric function: 3AFC B < A 1 A 2 < B A 2 B A 1

23 Psychometric function: 3AFC B < A 1 A 2 < B A 2 B A 1

24 Psychometric function: 3AFC B < A 1 B < A 2 B A 2 A 1

25 Psychometric function: 3AFC B < A 1 OR B < A 2

26 Psychometric function for 3AFC p c = P A 1 < B, A 2 < B Adapted from Christensen, R. H. B. (2012). Sensometrics: Thurstonian and Statistical Models. Technical University of Denmark, Kongens Lyngby, Denmark. PhD Thesis, p. 15.

27 Psychometric function for 3AFC p c = P A 1 < B, A 2 < B = න P A 1 < B, A 2 < B, B = z dz A δ B Adapted from Christensen, R. H. B. (2012). Sensometrics: Thurstonian and Statistical Models. Technical University of Denmark, Kongens Lyngby, Denmark. PhD Thesis, p. 15.

28 Psychometric function for 3AFC p c = P A 1 < B, A 2 < B = න = න P A 1 < B, A 2 < B, B = z dz P A 1 < z)p A 2 < z P(A 1 + δ = z dz A δ B Adapted from Christensen, R. H. B. (2012). Sensometrics: Thurstonian and Statistical Models. Technical University of Denmark, Kongens Lyngby, Denmark. PhD Thesis, p. 15.

29 Psychometric function for 3AFC p c = P A 1 < B, A 2 < B = න = න = න P A 1 < B, A 2 < B, B = z dz P A 1 < z)p A 2 < z P(A 1 + δ = z dz P(A 1 < z) 2 P A 1 = z δ dz Adapted from Christensen, R. H. B. (2012). Sensometrics: Thurstonian and Statistical Models. Technical University of Denmark, Kongens Lyngby, Denmark. PhD Thesis, p. 15.

30 Psychometric function for 3AFC p c = P A 1 < B, A 2 < B = න = න = න = න P A 1 < B, A 2 < B, B = z dz P A 1 < z)p A 2 < z P(A 1 + δ = z dz P(A 1 < z) 2 P A 1 = z δ dz ϕ z 2 φ z δ dz Adapted from Christensen, R. H. B. (2012). Sensometrics: Thurstonian and Statistical Models. Technical University of Denmark, Kongens Lyngby, Denmark. PhD Thesis, p. 15.

31 Psychometric function for 3AFC p c = P A 1 < B, A 2 < B = න = න = න = න P A 1 < B, A 2 < B, B = z dz P A 1 < z)p A 2 < z P(A 1 + δ = z dz P(A 1 < z) 2 P A 1 = z δ dz ϕ z 2 φ z δ dz cumulative distribution function Adapted from Christensen, R. H. B. (2012). Sensometrics: Thurstonian and Statistical Models. Technical University of Denmark, Kongens Lyngby, Denmark. PhD Thesis, p. 15.

32 Psychometric function for 3AFC p c = P A 1 < B, A 2 < B = න = න = න = න P A 1 < B, A 2 < B, B = z dz P A 1 < z)p A 2 < z P(A 1 + δ = z dz P(A 1 < z) 2 P A 1 = z δ dz ϕ z 2 φ z δ dz probability density function Adapted from Christensen, R. H. B. (2012). Sensometrics: Thurstonian and Statistical Models. Technical University of Denmark, Kongens Lyngby, Denmark. PhD Thesis, p. 15.

33 Psychometric function: 3AFC p c = න ϕ z 2 φ z δ dz It is possible to set δ, then solve for p c if δ = 0. 5 then p c = if δ = 1. 0 then p c = if δ = 1. 5 then p c = Adapted from Christensen, R. H. B. (2012). Sensometrics: Thurstonian and Statistical Models. Technical University of Denmark, Kongens Lyngby, Denmark. PhD Thesis, p. 15.

34 Psychometric function p c = f psy (dʹ) The R package sensr can be used to obtain dʹ estimates Christensen, R. H. B. & P. B. Brockhoff (2015). sensr - An R-package for sensory discrimination. R package version

35 Generalized linear models with a psychometric link function

36 GLM with Thurstonian Link Function Tetrad Xβ = f psy 1 δ = g f psy E.g. g tetrad p ij = β 0 + β j X i + ε ij

37 GLM with Thurstonian Link Function Original formulation vs. four prototypes vs.p1 vs.p2 vs.p3 vs.p4 A 6/19 8/19 11/19 13/19 B 12/19 13/19 16/19 17/19 where A and B are consumer segments Adapted from: Brockhoff, P. B. & Christensen, R. H. B. (2010). Thurstonian models for sensory discrimination tests as generalized linear models. Food Quality and Preference 21,

38 GLM with Thurstonian Link Function require(sensr) # tetrad data data <- expand.grid( conc = 1:4, segment = c("a", "B") ) data$correct <- c(6, 8, 11, 13, 12, 13, 16, 17) data$total <- rep(19, 8) # glm with appropriate thurstonian link function specified model <- glm( cbind(correct, total - correct) ~ segment + conc, data, family = tetrad ) family = tetrad Adapted from: Brockhoff, P. B. & Christensen, R. H. B. (2010). Thurstonian models for sensory discrimination tests as generalized linear models. Food Quality and Preference 21,

39 Analysis of Deviance (ANODE) table round(summary(model)$coefficients, 3) Estimate Std. Error z value Pr(> z ) (Intercept) segmentb conc Adapted from: Brockhoff, P. B. & Christensen, R. H. B. (2010). Thurstonian models for sensory discrimination tests as generalized linear models. Food Quality and Preference 21,

40 Analysis of Deviance (ANODE) table round(summary(model)$coefficients, 3) Estimate Std. Error z value Pr(> z ) (Intercept) segmentb conc Adapted from: Brockhoff, P. B. & Christensen, R. H. B. (2010). Thurstonian models for sensory discrimination tests as generalized linear models. Food Quality and Preference 21,

41 Analysis of Deviance (ANODE) table round(summary(model)$coefficients, 3) Estimate Std. Error z value Pr(> z ) (Intercept) segmentb conc Adapted from: Brockhoff, P. B. & Christensen, R. H. B. (2010). Thurstonian models for sensory discrimination tests as generalized linear models. Food Quality and Preference 21,

42 Analysis of Deviance (ANODE) table round(summary(model)$coefficients, 3) Estimate Std. Error z value Pr(> z ) (Intercept) segmentb conc Adapted from: Brockhoff, P. B. & Christensen, R. H. B. (2010). Thurstonian models for sensory discrimination tests as generalized linear models. Food Quality and Preference 21,

43 Confidence Intervals Brockhoff & Christensen (2010) also propose using likelihood intervals instead of Wald intervals (based on a normal approximation). This recommendation stands for all results, but seems especially important if p c 1.

44 d-prime calculation # compare with d-prime calculation from discrim method discrim(34, 55, method="threeafc", statistic="likelihood") Estimates for the threeafc discrimination protocol with 34 correct answers in 55 trials. One-sided p-value and 95 % two-sided confidence intervals are based on the likelihood root statistic. Estimate Std. Error Lower Upper pc pd d-prime Result of difference test: Likelihood Root statistic = , p-value: 8.099e-06 Alternative hypothesis: d-prime is greater than 0

45 d-prime calculation # compare with d-prime calculation from discrim method discrim(34, 55, method="threeafc", statistic="likelihood") Estimates for the threeafc discrimination protocol with 34 correct answers in 55 trials. One-sided p-value and 95 % two-sided confidence intervals are based on the likelihood root statistic. Estimate Std. Error Lower Upper pc pd d-prime Result of difference test: Likelihood Root statistic = , p-value: 8.099e-06 Alternative hypothesis: d-prime is greater than 0

46 Same-different test Thurstonian analysis Coefficients Estimate Std. Error Lower Upper P-value tau <2e-16 *** delta <2e-16 *** --- Signif. codes: 0 *** ** 0.01 * Log Likelihood: AIC:

47 Same-different test Thurstonian analysis dʹ Coefficients Estimate Std. Error Lower Upper P-value tau <2e-16 *** delta <2e-16 *** --- Signif. codes: 0 *** ** 0.01 * Log Likelihood: AIC:

48 Same-different test Thurstonian analysis τ dʹ Coefficients Estimate Std. Error Lower Upper P-value tau <2e-16 *** delta <2e-16 *** --- Signif. codes: 0 *** ** 0.01 * Log Likelihood: AIC:

49 dʹ =

50 τ = dʹ =

51 Rousseau and Ennis (2013) and Rousseau (2015) propose a strategy for determining consumer-relevant differences

52 Rousseau s strategy 1. Collect same-different data and obtain a good estimate of τ. 2. The reference for future decision-making is τ. 3. Consider the difference to be consumer-relevant if dʹ > τ. Otherwise, consider the difference to be non-consumer-relevant.

53 consumer relevant τ = dʹ =

54 not consumer relevant τ = dʹ =

55 So far the published evidence used to justify this strategy has been based on simulated data.

56 Equivalence study

57 Materials and Methods

58 psi=27 psi=35 psi=45 psi=65

59 Procedure Station for tetrad evaluation

60 Same-Different test Pair AB AC AD Possible Presentations AB AA BA BB AC AA CA CC AD AA DA DD BC BC BB CB CC BD CD BD BB DB DD CD CC DC DD

61 Tetrad test Pair Possible Presentations AB ABAB AABB BABA BBAA ABBA BAAB AC ACAC AACC CACA CCAA ACCA CAAC AD ADAD AADD DADA DDAA ADDA DAAD BC BCBC BBCC CBCB CCBB BCCB CBBC BD BDBD BBDD DBDB DDBB BDDB DBBD CD CDCD CCDD DCDC DDCC CDDC DCCD

62 2AC Preference test Pair AB AC AD BC BD CD AA BB CC DD Possible Presentations AB BA AC CA AD DA BC CB BD DB CD DC AA BB CC DD

63 Same-Different Tetrad 2AC Preference

64 not consumer relevant τ = dʹ = RECALL

65 τ from Same Different data τ AB AC AD BC BD CD

66 τ from Same Different data τ = 1.84 τ AB AC AD BC BD CD

67 τ = 1.84 relevant difference non-relevant difference relevant difference

68 τ = 1.84 relevant difference non-relevant difference relevant difference

69 τ = 1.84 relevant difference non-relevant difference relevant difference

70 Pair Samedifferent dʹ Consumer relevant? AB 1.67 No AC 3.32 Yes AD 5.18 Yes BC 2.77 Yes BD 5.29 Yes CD 2.95 Yes

71 Pair Samedifferent dʹ Consumer relevant? Tetrad dʹ Consumer relevant? AB 1.67 No 0.90 No AC 3.32 Yes 2.74 Yes AD 5.18 Yes 3.42 Yes BC 2.77 Yes 1.42 No BD 5.29 Yes 3.10 Yes CD 2.95 Yes 3.06 Yes

72 Same Different dʹ Tetrad dʹ dʹ AB AC AD BC BD CD

73 Same Different dʹ Tetrad dʹ dʹ AB AC AD BC BD CD

74 Tetrad dʹ Same Different dʹ 0 dʹ AB AC AD BC BD CD

75 Tetrad dʹ Same Different dʹ 0 dʹ AB AC AD BC BD CD

76 Conclusions consumer data simulated data Same Different dʹ Tetrad dʹ

77 Conclusions Tetrad dʹ = discriminal distance Reference: Castura & Franczak, 2017

78 Conclusions Tetrad dʹ = discriminal distance Same Different dʹ = conceptual distance Reference: Castura & Franczak, 2017

79 Conclusions τ from Same Different might not be method independent!

80 Conclusions For now, findings suggest that Rousseau s strategy for determining consumer relevance cannot be used across sensory method types

81 Conclusions Follow up studies are needed to confirm or disconfirm these findings.

82 Selected References Castura, J. C., & Franczak, B. C. (2017). Statistics for use in difference testing. In: L. Rogers (ed.), Discrimination Testing in Sensory Science - A Practical Handbook. Boca Raton, FL: Woodhead Publishing. (Forthcoming.) Christensen, R. H. B. (2012). Sensometrics: Thurstonian and Statistical Models. Technical University of Denmark, Kongens Lyngby, Denmark. PhD Thesis. Christensen, R. H. B., & Brockhoff, P. B. (2016). sensr - An R-package for sensory discrimination. R package version Rousseau, B. (2015). Sensory discrimination testing and consumer relevance. Food Quality and Preference, 43, Rousseau, B., & Ennis, D. M. (2013). When Are Two Products Close Enough to be Equivalent? IFPress, 16(1), 3-4.

83 Thank you for your kind attention John C. Castura Compusense Inc. Guelph, ON, Canada