What are the requirements of a listening panel for evaluating spatial audio quality ?

This paper addresses the topic of subject and listening panel performance in descriptive analysis applications. A summary of the status in the field of audio is provided regarding the selection, development, training and monitoring of subjects and listening panel. Subject categorisation and associated terminologies are discussed, placing emphasis on the difference between affective and descriptive tasks. A number of statistical analysis methods are presented that can be applied to the evaluation of subject and a panel’s performance in descriptive analysis tasks which are supported by some illustrative examples. Additionally, the application of these analysis methods to spatial sound evaluation tasks is discussed.

[1]  R. Marshall,et al.  SENSORY MEASUREMENT OF FOOD TEXTURE BY FREE‐CHOICE PROFILING , 1988 .

[2]  TWO TYPES OF SENSORY PANELS OR ARE THERE MORE , 2005 .

[3]  Wieslaw Woszczyk,et al.  A computer-assisted method for training and researching timbre memory and evaluation skills , 2002 .

[4]  J. Mandel.  The validation of measurement through interlaboratory studies , 1991 .

[5]  Andrzej Miskiewicz.  Timbre Solfege: A Course in Technical Listening for Sound Engineers , 1992 .

[6]  Søren Bech.  Selection and Training of Subjects for Listening Tests on Sound-Reproducing Equipment , 1992 .

[7]  Jean A. McEwan,et al.  An application of the repertory grid method to investigate consumer perceptions of foods , 1988, Appetite.

[8]  Francis Rumsey,et al.  Verification and correlation of attributes used for describing the spatial quality of reproduced sound , 2001 .

[9]  Jens Blauert,et al.  Concepts Behind Sound Quality: Some Basic Considerations , 2003 .

[10]  Herbert Stone,et al.  Sensory Evaluation Practices , 1985 .

[11]  G. Kelly.  The Psychology of Personal Constructs , 2020 .

[12]  Gillian M. Arnold,et al.  A comparison of the aromas of six coffees characterised by conventional profiling, free‐choice profiling and similarity scaling methods , 1985 .

[13]  Sean Olive.  A Method for Training Listeners and Selecting Program Material for Listening Tests , 1994 .

[14]  Per M. Brockhoff,et al.  Modelling individual differences between assessors in sensory evaluations , 1994 .

[15]  Tormod Næs,et al.  Measuring validity in sensory analysis , 1995 .

[16]  David Isherwood,et al.  Augmentation, Application and Verification of the Generalized Listener Selection Procedure , 2003 .

[17]  Francis Rumsey,et al.  Systematic Evaluation of Perceived Spatial Quality , 2003 .

[18]  T. Næs,et al.  Multivariate analysis of data in sensory science , 1996 .

[19]  M. Martens,et al.  SENSORY-RHEOLOGICAL RELATIONSHIPS IN INSTANT HOT COCOA DRINKS , 1999 .

[20]  Francis Rumsey,et al.  Identification of Perceived Spatial Attributes of Recordings by Repertory Grid Technique and Other Methods , 1999 .

[21]  D. V. Byrne,et al.  Sensory panel consistency during development of a vocabulary for warmed-over flavour , 2001 .

[22]  Francis Rumsey,et al.  Training of Listeners for the Evaluation of Spatial Sound Reproduction , 2002 .

[23]  Wieslaw Woszczyk,et al.  A Computer-Aided System for Timbral Ear Training , 1994 .

[24]  Jean A. McEwan,et al.  Proficiency testing for sensory profile panels: measuring panel performance , 2002 .

[25]  Garmt Dijksterhuis,et al.  Assessing panel consonance , 1995 .

[26]  Nick Zacharov,et al.  Unravelling the Perception of Spatial Sound Reproduction: Analysis & External Preference Mapping , 2001 .

[27]  Tormod Næs,et al.  Detection and interpretation of variation within and between assessors in sensory profiling , 1991 .

[28]  John C. Castura,et al.  Use of feedback calibration to reduce the training time for wine panels , 2006 .

[29]  Tobias Neher,et al.  Towards a spatial ear trainer. , 2004 .

[30]  Tormod Næs,et al.  Analysis of Variance for Sensory Data , 1997 .

[31]  Jian Bi,et al.  AGREEMENT AND RELIABILITY ASSESSMENTS FOR PERFORMANCE OF SENSORY DESCRIPTIVE PANEL , 2003 .

[32]  Francis Rumsey,et al.  In Search of the Spatial Dimensions of Reproduced Sound: Verbal Protocol Analysis and Cluster Analysis of Scaled Verbal Descriptors , 2000 .

[33]  Sébastien Lê,et al.  Confidence ellipse for the sensory profiles obtained by principal component analysis , 2005 .

[34]  J. Gower.  Generalized procrustes analysis , 1975 .

[35]  J. Delarue,et al.  Sensory mapping using Flash profile. Comparison with a conventional descriptive method for the evaluation of the flavour of fruit dairy products , 2004 .

[36]  Pascal Schlich.  GRAPES: A METHOD AND A SAS® PROGRAM FOR GRAPHICAL REPRESENTATIONS OF ASSESSOR PERFORMANCES , 1994 .

[37]  Anthony A. Williams,et al.  The use of free-choice profiling for the evaluation of commercial ports , 1984 .

[38]  Søren Bech,et al.  Perceptual Audio Evaluation - Theory, Method and Application , 2006 .

[39]  Gaetan Lorho,et al.  Individual Vocabulary Profiling of Spatial Enhancement Systems for Stereo Headphone Reproduction , 2005 .

[40]  Jason Corey.  An Ear Training System for Identifying Parameters of Artificial Reverberation in Multichannel Audio , 2004 .

[41]  H. Martens,et al.  Multivariate analysis of quality , 2000 .

[42]  Sean Olive.  A New Listener Training Software Application , 2001 .

[43]  Andreas Rytz,et al.  Training is a critical step to obtain reliable product profiles in a real food industry context , 2004 .

[44]  Juha Merimaa,et al.  Training of Listeners for Evaluation of Spatial Attributes of Sound , 2004 .

[45]  René Quesnel.  Timbral Ear Trainer: Adaptive, Interactive Training of Listening Skills for Evaluation of Timbre , 1996 .

[46]  A. A. Williams,et al.  Sensory quality in foods and beverages : definition, measurement and control , 1983 .

[47]  Pascal Schlich,et al.  Defining and Validating Assessor Compromises About Product Distances and Attribute Correlations , 1996 .

[48]  Thierry Couronne,et al.  A study of assessors' performance using graphical methods , 1997 .

[49]  Frank Rossi,et al.  Assessing sensory panelist performance using repeatability and reproducibility measures , 2001 .

[50]  Francis Rumsey,et al.  Spatial Attribute Identification and Scaling by Repertory Grid Technique and Other Methods , 1999 .

[51]  El Mostafa Qannari,et al.  Assessment of the agreement among the subjects in fixed vocabulary profiling , 2006 .

[52]  M. Martens,et al.  Analysis of sensory assessors in texture profiling of potatoes by multivariate modelling , 2000 .

[53]  C. J. Wolters,et al.  Effect of training procedure on the performance of descriptive panels , 1994 .

[54]  Roger Shively,et al.  Listener Training and Repeatability for Automobiles , 1998 .

[55]  Tomasz Letowski.  Development of Technical Listening Skills: Timbre Solfeggio , 1985 .

[56]  Morten Meilgaard,et al.  Sensory Evaluation Techniques , 2020 .

[57]  Per B. Brockhoff,et al.  Statistical testing of individual differences in sensory profiling , 2003 .

[58]  Francis Rumsey,et al.  Correlation between Emotive, Descriptive and Naturalness Attributes in Subjective Data Relating to Spatial Sound Reproduction , 2000 .

[59]  Nick Zacharov,et al.  Unravelling the Perception of Spatial Sound Reproduction: Language Development, Verbal Protocol Analysis and Listener Training , 2001 .