Acoustic Scene Classification Using Multichannel Observation with Partially Missing Channels

Sounds recorded with smartphones or IoT devices often have partially unreliable observations caused by clipping, wind noise, and completely missing parts due to microphone failure and packet loss in data transmission over the network. In this paper, we investigate the impact of the partially missing channels on the performance of acoustic scene classification using multichannel audio recordings, especially for a distributed microphone array. Missing observations cause not only losses of time-frequency and spatial information on sound sources but also a mismatch between a trained model and evaluation data. We thus investigate how a missing channel affects the performance of acoustic scene classification in detail. We also propose simple data augmentation methods for scene classification using multichannel observations with partially missing channels and evaluate the scene classification performance using the data augmentation methods.

[1]  Florian Metze,et al.  Event-based Video Retrieval Using Audio , 2012, INTERSPEECH.

[2]  Xinxing Chen,et al.  Acoustic scene classification using multi-scale features , 2018, DCASE.

[3]  Visar Berisha,et al.  A sensor network for real-time acoustic scene analysis , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[4]  Tuomas Virtanen,et al.  Acoustic event detection in real life recordings , 2010, 2010 18th European Signal Processing Conference.

[5]  Gernot A. Fink,et al.  BAG-OF-FEATURES ACOUSTIC EVENT DETECTION FOR SENSOR NETWORKS , 2016 .

[6]  Keisuke Imoto,et al.  Introduction to acoustic event and scene analysis , 2018 .

[7]  Ankit Shah,et al.  DCASE2017 Challenge Setup: Tasks, Datasets and Baseline System , 2017, DCASE.

[8]  Nobutaka Ono,et al.  Acoustic Topic Model for Scene Analysis With Intermittently Missing Observations , 2019, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[9]  Stefano Squartini,et al.  A convolutional neural network approach for acoustic scene classification , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[10]  Nikos Fakotakis,et al.  On acoustic surveillance of hazardous situations , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[11]  Bart Vanrumste,et al.  DCASE 2018 Challenge - Task 5: Monitoring of domestic activities based on multi-channel acoustics , 2018, ArXiv.

[12]  Fuchun Sun,et al.  AN ENSEMBLE SYSTEM FOR DOMESTIC ACTIVITY RECOGNITION Technical Report , 2018 .

[13]  Marian Verhelst,et al.  The SINS Database for Detection of Daily Activities in a Home Environment Using an Acoustic Sensor Network , 2017, DCASE.

[14]  Nobutaka Ono,et al.  Spatial Cepstrum as a Spatial Feature Using a Distributed Microphone Array for Acoustic Scene Analysis , 2017, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[15]  Petros Maragos,et al.  Multi-room speech activity detection using a distributed microphone network in domestic environments , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[16]  Keisuke Imoto.  Graph Cepstrum: Spatial Feature Extracted from Partially Connected Microphones , 2020, IEICE Trans. Inf. Syst..

[17]  Vesa T. Peltonen,et al.  Audio-based context recognition , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[18]  Suehiro Shimauchi,et al.  Acoustic Scene Analysis Based on Hierarchical Generative Model of Acoustic Event Sequence , 2016, IEICE Trans. Inf. Syst..

[19]  Liyuan Liu,et al.  On the Variance of the Adaptive Learning Rate and Beyond , 2019, ICLR.

[20]  Kyogu Lee,et al.  Convolutional Neural Networks with Binaural Representations and Background Subtraction for Acoustic Scene Classification , 2017, DCASE.

[21]  S. Y. Ezra,et al.  MULTICHANNEL AUDIO CLASSIFICATION WITH NEURAL NETWORKS USING SCATTERING TRANSFORM , 2018 .

[22]  Shiqiang Wang,et al.  DOMESTIC ACTIVITIES CLASSIFICATION BASED ON CNN USING SHUFFLING AND MIXING DATA AUGMENTATION Technical Report , 2018 .

[23]  Tuomas Virtanen,et al.  ACOUSTIC SCENE CLASSIFICATION USING CONVOLUTIONAL RECURRENT NEURAL NETWORKS , 2017 .

[24]  Mark D. Plumbley,et al.  Computational Analysis of Sound Scenes and Events , 2017 .

[25]  Annamaria Mesaros,et al.  Acoustic Scene Classification in DCASE 2020 Challenge: Generalization Across Devices and Low Complexity Solutions , 2020, DCASE.