Perceptual Evaluation of Synthesized Sound Effects

Sound synthesis is the process of generating artificial sounds through some form of simulation or modelling. This article aims to identify which sound synthesis methods achieve the goal of producing a believable audio sample that may replace a recorded sound sample. A perceptual evaluation experiment of five different sound synthesis techniques was undertaken. Additive synthesis, statistical modelling synthesis with two different feature sets, physically inspired synthesis, concatenative synthesis, and sinusoidal modelling synthesis were all compared. Evaluation using eight different sound class stimuli and 66 different samples was undertaken. The additive synthesizer is the only synthesis method not considered significantly different from the reference sample across all sounds classes. The results demonstrate that sound synthesis can be considered as realistic as a recorded sample and makes recommendations for use of synthesis methods, given different sound class contexts.

[1]  R. Caussé,et al.  The representation of auditory source characteristics: simple geometric form. , 1996, Perception & psychophysics.

[2]  J. Ballas.  Common factors in the identification of an assortment of brief everyday sounds. , 1993, Journal of experimental psychology. Human perception and performance.

[3]  Axel Röbel,et al.  A Montage Approach to Sound Texture Synthesis , 2016, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[4]  Diemo Schwarz,et al.  State of the Art in Sound Texture Synthesis , 2011 .

[5]  RECOMMENDATION ITU-R BS.1387-1 - Method for objective measurements of perceived audio quality , 2002 .

[6]  Davide Rocchesso,et al.  The Sounding Object , 2002 .

[7]  Andrew Horner,et al.  Sound Texture Synthesis Using an Overlap–Add/Granular Synthesis Approach , 2009 .

[8]  Mathieu Lagrange,et al.  Perceptual Evaluation of a Real-time Synthesis Technique for Rolling Sounds , 2007 .

[9]  Diemo Schwarz,et al.  Concatenative Sound Texture Synthesis Methods and Evaluation , 2016 .

[10]  Joshua D. Reiss,et al.  Physically Derived Sound Synthesis Model of a Propeller , 2017, Audio Mostly Conference.

[11]  Thomas Sporer,et al.  PEAQ - The ITU Standard for Objective Measurement of Perceived Audio Quality , 2000 .

[12]  Andrew Horner,et al.  Evaluation of Iterative Matching for Scalable Wavetable Synthesis , 2006 .

[13]  Stefan Bilbao,et al.  Finite difference time domain simulation for the brass instrument bore. , 2013, The Journal of the Acoustical Society of America.

[14]  Joshua D. Reiss,et al.  Sound Synthesis of Objects Swinging through Air Using Physical Models , 2017 .

[15]  Richard Kronland-Martinet,et al.  Perceptual characterization of motion evoked by sounds for synthesis control purposes , 2013, TAP.

[16]  Stefania Serafin,et al.  DESIGN AND EVALUATION OF PHYSICALLY INSPIRED MODELS OF SOUND EFFECTS IN COMPUTER GAMES , 2009 .

[17]  Joshua D. Reiss,et al.  Unsupervised Taxonomy of Sound Effects , 2017 .

[18]  Xavier Serra,et al.  Essentia: An Audio Analysis Library for Music Information Retrieval , 2013, ISMIR.

[19]  MoffatDavid,et al.  Perceptual Evaluation of Synthesized Sound Effects , 2018 .

[20]  Emmanuel Ifeachor,et al.  Objective Prediction of Sound Synthesis Quality , 2003 .

[21]  Emmanuel Ifeachor,et al.  Perceptual Modeling of Piano Tones , 2005 .

[22]  David A. Jaffe.  Ten Criteria for Evaluating Synthesis Techniques , 1995 .

[23]  Perry R. Cook,et al.  PERCEPTUAL SPACES FOR SOUND EFFECTS OBTAINED WITH AN INTERACTIVE SIMILARITY RATING PROGRAM , 2001 .

[24]  Stefan Bilbao.  Numerical Sound Synthesis: Finite Difference Schemes and Simulation in Musical Acoustics , 2009 .

[25]  Perry R. Cook,et al.  Real Sound Synthesis for Interactive Applications , 2002 .

[26]  Eero P. Simoncelli,et al.  Sound texture synthesis via filter statistics , 2009, 2009 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics.

[27]  Timothy E. Goldsmith,et al.  Data collection and analysis techniques for evaluating the perceptual qualities of auditory stimuli , 1998, TAP.

[28]  Andy Farnell,et al.  Designing Sound , 2008 .

[29]  Patrick Susini,et al.  The Role of Sound Source Perception in Gestural Sound Description , 2014, TAP.

[30]  Joshua D. Reiss,et al.  An Evaluation of Audio Feature Extraction Toolboxes , 2015 .

[31]  Cumhur Erkut,et al.  Synthesis of Hand Clapping Sounds , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[32]  Christiane Fellbaum,et al.  SoundNet: investigating a language composed of environmental sounds , 2010, CHI.

[33]  David Moffat,et al.  Web Audio Evaluation Tool: A Browser-based Listening Test Environment , 2015 .

[34]  Richard Kronland-Martinet,et al.  A 3-D Immersive Synthesizer for Environmental Sounds , 2010, IEEE Transactions on Audio, Speech, and Language Processing.

[35]  Method for the subjective assessment of intermediate quality level of , 2014 .

[36]  Joshua D. Reiss,et al.  REAL-TIME PHYSICAL MODEL FOR SYNTHESIS OF SWORD SWING SOUNDS , 2017 .

[37]  Davide Rocchesso,et al.  Sounding Objects , 2003, IEEE Multim..

[38]  Vytis Puronas.  Sonic hyperrealism: illusions of a non-existent aural reality , 2014 .

[39]  Perry R. Cook,et al.  Feature-Based Synthesis: Mapping Acoustic and Perceptual Features onto Synthesis Parameters , 2006, ICMC.

[40]  Thomas P. Caudell,et al.  Using wavelets to synthesize stochastic-based sounds for immersive virtual environments , 2005, TAP.

[41]  Henrik Hahn,et al.  Expressive Sampling Synthesis - Learning Extended Source-Filter Models from Instrument Sound Databases for Expressive Sample Manipulations. (Synthèse et transformation des sons basés sur les modèles de type source-filtre étendu pour les instruments de musique) , 2015 .

[42]  Luca Turchet,et al.  Sound synthesis and evaluation of interactive footsteps for virtual reality applications , 2010, 2010 IEEE Virtual Reality Conference (VR).

[43]  Joshua D. Reiss,et al.  Real-time physical model of an Aeolian harp , 2017 .

[44]  Jörn Loviscach,et al.  Automatic Cloning of Recorded Sounds by Software Synthesizers , 2009 .

[45]  Julius O. Smith,et al.  Spectral modeling synthesis: A sound analysis/synthesis based on a deterministic plus stochastic decomposition , 1990 .

[46]  Joshua D. Reiss,et al.  Physical Modeling and Synthesis of Motor Noise for Replication of a Sound Effects Library , 2010 .

[47]  Richard Kronland-Martinet,et al.  Perceptual Control of Environmental Sound Synthesis , 2011, CMMR/FRSM.

[48]  Andrew P. McPherson,et al.  Mapping and interaction strategies for performing environmental sound , 2014, 2014 IEEE VR Workshop: Sonic Interaction in Virtual Environments (SIVE).

[49]  Stefania Serafin,et al.  Procedural Audio in Computer Games Using Motion Controllers: An Evaluation on the Effect and Perception , 2013, Int. J. Comput. Games Technol..

[50]  Matti Karjalainen,et al.  Evaluation of Modern Sound Synthesis Methods , 1998 .

[51]  Joshua D. Reiss,et al.  Creating Real-Time Aeroacoustic Sound Effects Using Physically Informed Models , 2018 .

[52]  Brecht De Man,et al.  Web Audio Evaluation Tool: A framework for subjective assessment of audio , 2016 .

[53]  Eero P. Simoncelli,et al.  Article Sound Texture Perception via Statistics of the Auditory Periphery: Evidence from Sound Synthesis , 2022 .

[54]  LUCAS MENGUAL,et al.  Modal Synthesis of Weapon Sounds , 2016 .

[55]  Juan Pampin.  ATS: A System for Sound Analysis Transformation and Synthesis Based on a Sinusoidal plus Critical-Band Noise Model and Psychoacoustics , 2004, ICMC.

[56]  Vesa Välimäki,et al.  Perception and Adjustment of Pitch in Inharmonic String Instrument Tones , 2002 .

[57]  Stefan Bilbao,et al.  Numerical Sound Synthesis , 2009 .

[58]  Vesa Välimäki,et al.  A Subjective Validation Method for Musical Instrument Emulation , 2011 .

[59]  Richard Kronland-Martinet,et al.  Abstract Sounds and Their Applications in Audio and Perception Research , 2010, CMMR.

[60]  Perry R. Cook,et al.  Toward Synthesized Environments: A Survey of Analysis and Synthesis Methods for Sound Designers and Composers , 2009, ICMC.

[61]  Perry R. Cook,et al.  Feature-Based Synthesis: A Tool for Evaluating, Designing, and Interacting with Music IR Systems , 2006, ISMIR.

[62]  Xavier Serra,et al.  A sound analysis/synthesis system based on a deterministic plus stochastic decomposition , 1990 .