A method to investigate the stability of a sorting map

Abstract Many statistical tools are available to check the performance of descriptive analysis panels (alignment, repeatability, and discrimination). There are no equivalent concepts for sorting data. In particular, there is currently no way to clearly assess if the results given by a panel are stable. The main goal of this study is to introduce an indicator of stability of a sorting map, the RVb coefficient, and to assess its relevance. The calculation of the RVb is based on a resampling approach. The stability of sorting maps was assessed using the RVb coefficient on five data sets varying in terms of number of products (8–15) and number of evaluations (10–118). As hypothesized, the RVb coefficient increased with the number of evaluations, decreased with the complexity of the sensory task, and increased with the expertise level of the panelists (which significantly impacted both the repeatability and homogeneity). Therefore, the RVb coefficient was found to be a relevant indicator of stability. A strong recommendation for future sorting studies is to systematically check the stability of the sorting map after data was collected. This can easily be achieved with the RVb coefficient.

[1]  Jérôme Pagès,et al.  Collection and analysis of perceived product inter-distances using multiple factor analysis: Application to the study of 10 white wines from the Loire Valley , 2005 .

[2]  Andreas Rytz,et al.  Sorting procedure as an alternative to quantitative descriptive analysis to obtain a product sensory map , 2006 .

[3]  Harry T. Lawless,et al.  Perceptual mapping of apples and cheeses using projective mapping and sorting. , 2010 .

[4]  J. Shao,et al.  The jackknife and bootstrap , 1996 .

[5]  Dominique Valentin,et al.  Analyzing assessors and products in sorting tasks: DISTATIS, theory and applications , 2007 .

[6]  Jérôme Pagès,et al.  Inter-laboratory comparison of sensory profiles: methodology and results , 2001 .

[7]  Dominique Valentin,et al.  IMPACT OF TRAINING ON BEER FLAVOR PERCEPTION AND DESCRIPTION: ARE TRAINED AND UNTRAINED SUBJECTS REALLY DIFFERENT? , 2001 .

[8]  Pascal Schlich,et al.  SORT-CC: A procedure for the statistical treatment of free sorting data , 2010 .

[9]  Y. Escoufier.  LE TRAITEMENT DES VARIABLES VECTORIELLES , 1973 .

[10]  Dominique Valentin,et al.  Beer-Trained and Untrained Assessors Rely More on Vision than on Taste When They Categorize Beers , 2009 .

[11]  Pauline Faye,et al.  Sensory evaluation of automotive fabrics: the contribution of categorization tasks and non verbal information to set-up a descriptive method of tactile properties , 2001 .

[12]  Pascal Schlich,et al.  MODELING THE EVOLUTION OF THE PERFORMANCE OF A SENSORY PANEL: A MIXED‐MODEL AND CONTROL CHART APPROACH , 2007 .

[13]  Damien Brémaud,et al.  An alternative to external preference mapping based on consumer perceptive mapping , 2006 .

[14]  Einar Risvik,et al.  Projective mapping: A tool for sensory analysis and consumer research , 1994 .

[15]  Pascal Schlich,et al.  Sensory methodologies and the taste of water. , 2010 .

[16]  H. Lawless.  Exploration of fragrance categories and ambiguous odors using multidimensional scaling and cluster analysis , 1989 .

[17]  Pascal Schlich.  GRAPES: A METHOD AND A SAS® PROGRAM FOR GRAPHICAL REPRESENTATIONS OF ASSESSOR PERFORMANCES , 1994 .

[18]  Sébastien Lê,et al.  SENSOMINER : A PACKAGE FOR SENSORY DATA ANALYSIS , 2008 .

[19]  Sébastien Lê,et al.  Confidence ellipse for the sensory profiles obtained by principal component analysis , 2005 .

[20]  Ivanne Soufflet,et al.  A comparison between industrial experts' and novices' haptic perceptual organization: a tool to identify descriptors of the handle of fabrics , 2004 .

[21]  Sébastien Lê,et al.  A Factorial Approach for Sorting Task data (FAST) , 2009 .

[22]  N. Mantel.  The detection of disease clustering and a generalized regression approach. , 1967, Cancer research.

[23]  Michel Tenenhaus,et al.  Measurement of the reliability of sensory panel performances , 2006 .

[24]  A. W. MacRae,et al.  Perceptual variation among drinking waters: the reliability of sorting and ranking data for multidimensional scaling , 1997 .

[25]  P. Howgate,et al.  Assessing the similarity of odours by sorting and by triadic comparison , 1990 .

[26]  Sylvie Chollet,et al.  Comparison of visual appearance and texture profiles of jellies in France and Vietnam and validation of attribute transfer between the two countries , 2008 .

[27]  Juyun Lim,et al.  Qualitative differences of divalent salts: multidimensional scaling and cluster analysis. , 2005, Chemical senses.

[28]  Frank Rossi,et al.  Assessing sensory panelist performance using repeatability and reproducibility measures , 2001 .

[29]  Hervé Abdi,et al.  A modified sorting task to investigate consumer perceptions of extra virgin olive oils , 2010 .

[30]  Hervé Abdi,et al.  How to compute reliability estimates and display confidence and tolerance intervals for pattern classifiers using the Bootstrap and 3-way multidimensional scaling (DISTATIS) , 2009, NeuroImage.

[31]  Pauline Faye,et al.  Perceptive free sorting and verbalization tasks with naive subjects: an alternative to descriptive mappings , 2004 .

[32]  Harry T. Lawless,et al.  Multidimensional scaling of sorting data applied to cheese perception , 1995 .

[33]  李幼升,et al.  Ph , 1989 .

[34]  G. Blancher,et al.  French and Vietnamese: How do they describe texture characteristics of the same food? A case study with jellies , 2007 .

[35]  Christian Hennig,et al.  Cluster-wise assessment of cluster stability , 2007, Comput. Stat. Data Anal..

[36]  Harry T. Lawless,et al.  CONSISTENCY OF MULTIDIMENSIONAL SCALING MODELS DERIVED FROM ODOR SORTING , 1990 .