A Framework for Automatic Ontology Generation Based on Semantic Audio Analysis

Ontologies have been established for knowledge sharing and are widely used for structuring domains of interests conceptually. With growing amount of data on the internet, manual annotation and development of ontologies becomes critical. We propose a hybrid system to develop ontologies from audio signals automatically, in order to provide assistance to ontology engineers. The method is examined using various musical instruments, from wind and string families, that are classified using timbre features extracted from audio. To obtain models of the analysed instrument recordings, we use K-means clustering and determine an optimised codebook of Line Spectral Frequencies (LSFs) or Mel-frequency Cepstral Coefficients (MFCCs). The system was tested using two classification techniques, Multi-Layer Perceptron (MLP) neural network and Support Vector Machines (SVM). We then apply Formal Concept Analysis (FCA) to derive a lattice of concepts which is transformed into an ontology using the Ontology Web Language (OWL). The system was evaluated using Multivariate Analysis of Variance (MANOVA), with the feature and classifier attributes as independent variables and the lexical and taxonomic evaluation metrics as dependent variables.

[1]  Jim Matson,et al.  A New Approach to the Classification of Sound-Producing Instruments , 1985 .

[2]  Bernhard Ganter,et al.  Formal Concept Analysis, 6th International Conference, ICFCA 2008, Montreal, Canada, February 25-28, 2008, Proceedings , 2008, International Conference on Formal Concept Analysis.

[3]  Grigorios Tsoumakas,et al.  Multi-Label Classification of Music into Emotions , 2008, ISMIR.

[4]  R. Kronland-Martinet,et al.  From Clarinet Control to Timbre Perception , 2010 .

[5]  Nicola Guarino,et al.  An Overview of OntoClean , 2004, Handbook on Ontologies.

[6]  John F. Sowa,et al.  Knowledge representation: logical, philosophical, and computational foundations , 2000 .

[7]  Mark Sandler,et al.  Knowledge Management On The Semantic Web: A Comparison of Neuro-Fuzzy and Multi-Layer Perceptron Methods For Automatic Music Tagging , 2012 .

[8]  David J. Israel,et al.  A Question-Answering System for AP Chemistry: Assessing KR&R Technologies , 2004, KR.

[9]  D. Vasumathi,et al.  EFFICIENT WEB USAGE MINING BASED ON FORMAL CONCEPT ANALYSIS , 2009 .

[10]  Mark B. Sandler,et al.  Music Information Retrieval Using Social Tags and Audio , 2009, IEEE Transactions on Multimedia.

[11]  Thomas R. Gruber,et al.  A translation approach to portable ontology specifications , 1993, Knowl. Acquis..

[12]  Chin-Hui Lee,et al.  On the asymptotic statistical behavior of empirical cepstral coefficients , 1993, IEEE Trans. Signal Process..

[13]  Nacéra Bennacer,et al.  Ontology Discovery from Web Pages : Application to Tourism , 2004 .

[14]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[15]  Huajun Chen,et al.  The Semantic Web , 2011, Lecture Notes in Computer Science.

[16]  York Sure-Vetter,et al.  Automatic Evaluation of Ontologies (AEON) , 2005, SEMWEB.

[17]  G. Peeters Automatic Classification of Large Musical Instrument Databases Using Hierarchical Classifiers with Inertia Ratio Maximization , 2003 .

[18]  Nicola Guarino,et al.  A Formal Ontology of Properties , 2000, EKAW.

[19]  D6.6.1 Report on the integration of ML, HLT and OM , 2005 .

[20]  Steffen Staab,et al.  Learning Concept Hierarchies from Text Corpora using Formal Concept Analysis , 2005, J. Artif. Intell. Res..

[21]  Biing-Hwang Juang,et al.  A vector quantization approach to speaker recognition , 1985, ICASSP '85. IEEE International Conference on Acoustics, Speech, and Signal Processing.

[22]  Andreas Stolcke,et al.  Acoustic front-end optimization for bird species recognition , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[23]  Ichiro Fujinaga,et al.  Implementation of exemplar-based learning model for music cognition , 1998 .

[24]  Marcel Bauer Perception And The Senses , 2016 .

[25]  Rudolf Wille,et al.  Restructuring Lattice Theory: An Approach Based on Hierarchies of Concepts , 2009, ICFCA.

[26]  Steffen Staab,et al.  On How to Perform a Gold Standard Based Evaluation of Ontology Learning , 2006, SEMWEB.

[27]  M. K. Gill,et al.  Vector Quantization based Speaker Identification , 2010 .

[28]  Gilad Mishne,et al.  Learning domain ontologies for Web service descriptions: an experiment in bioinformatics , 2005, WWW '05.

[29]  Masataka Goto,et al.  Musical instrument identification based on F0-dependent multivariate normal distribution , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[30]  Tuomas Virtanen,et al.  Separation of drums from polyphonic music using non-negative matrix factorization and support vector machine , 2005, 2005 13th European Signal Processing Conference.

[31]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[32]  Hans Heinz Dräger,et al.  Prinzip einer Systematik der Musikinstrumente , 1950 .

[33]  Gaël Richard,et al.  Musical instrument recognition by pairwise classification strategies , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[34]  K. Paliwal,et al.  Efficient vector quantization of LPC parameters at 24 bits/frame , 1990 .

[35]  Suk-Hyung Hwang,et al.  FCA-based Conceptual Knowledge Discovery in Folksonomy , 2009 .

[36]  Nacéra Bennacer,et al.  A framework for retrieving conceptual knowledge from Web pages , 2005, SWAP.

[37]  Paul Lamere,et al.  Social Tagging and Music Information Retrieval , 2008 .

[38]  Steffen Staab,et al.  COMM: Designing a Well-Founded Multimedia Ontology for the Web , 2007, ISWC/ASWC.

[39]  Perry R. Cook,et al.  Easy As CBA: A Simple Probabilistic Model for Tagging Music , 2009, ISMIR.

[40]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[41]  Petko Valtchev,et al.  Formal Concept Analysis-Based Class Hierarchy Design in Object-Oriented Software Development , 2005, Formal Concept Analysis.

[42]  Michael Devitt,et al.  The Blackwell Guide to the Philosophy of Language , 2006 .

[43]  Mark Sandler,et al.  On the Effect of Reverberation on Musical Instrument Automatic Recognition , 2010 .

[44]  Mark Levy,et al.  Lightweight measures for timbral similarity of musical audio , 2006, AMCMM '06.

[45]  Chang-Hsing Lee,et al.  Automatic Recognition of Bird Songs Using Cepstral Coefficients , 2006 .

[46]  Gregor Snelting Concept Lattices in Software Analysis , 2005, Formal Concept Analysis.

[47]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[48]  Peter Kabal,et al.  The computation of line spectral frequencies using Chebyshev polynomials , 1986, IEEE Trans. Acoust. Speech Signal Process..

[49]  Steffen Staab,et al.  Measuring Similarity between Ontologies , 2002, EKAW.

[50]  Kaare Brandt Petersen,et al.  Mel Frequency Cepstral Coefficients: An Evaluation of Robustness of MP3 Encoded Music , 2006, ISMIR.

[51]  Lie Lu,et al.  Digital Object Identifier (DOI) 10.1007/s00530-002-0065-0 Multimedia Systems , 2003 .

[52]  Seppo Ilmari Fagerlund,et al.  Bird Species Recognition Using Support Vector Machines , 2007, EURASIP J. Adv. Signal Process..

[53]  Giancarlo Guizzardi,et al.  On Ontology, ontologies, Conceptualizations, Modeling Languages, and (Meta)Models , 2007, DB&IS.

[54]  Steffen Staab,et al.  Ontology Learning for the Semantic Web , 2002, IEEE Intell. Syst..

[55]  Gaël Richard,et al.  Drum Loops Retrieval from Spoken Queries , 2005, Journal of Intelligent Information Systems.

[56]  Jérôme Euzenat,et al.  Semantic Precision and Recall for Ontology Alignment Evaluation , 2007, IJCAI.

[57]  Tao Li,et al.  A comparative study on content-based music genre classification , 2003, SIGIR.

[58]  Thomas Keays,et al.  Semantic Web for the Working Ontologist , 2008 .

[59]  I. A. Richards,et al.  The Meaning of Meaning: a Study of the Influence of Language upon Thought and of the Science of Symbolism , 1923, Nature.

[60]  Harry F. Olson,et al.  Music, Physics and Engineering , 1967 .

[61]  David De Roure,et al.  An Ontological Framework for Semantic Description of Devices , 2004 .

[62]  Ales Leonardis,et al.  Incremental PCA for on-line visual learning and recognition , 2002, Object recognition supported by user interaction for service robots.

[63]  Arthur Gretton,et al.  An online support vector machine for abnormal events detection , 2006, Signal Process..

[64]  Stephen R. Marsland,et al.  Machine Learning - An Algorithmic Perspective , 2009, Chapman and Hall / CRC machine learning and pattern recognition series.

[65]  Asunción Gómez-Pérez,et al.  ONTOMETRIC: A Method to Choose the Appropriate Ontology , 2004, J. Database Manag..

[66]  Mohammad Bagher Menhaj,et al.  Training feedforward networks with the Marquardt algorithm , 1994, IEEE Trans. Neural Networks.

[67]  Xavier Rodet,et al.  Instrument identification in solo and ensemble music using Independent Subspace Analysis , 2004, ISMIR.

[68]  Lotfi Lakhal,et al.  iO2 - An Algorithmic Method for Building Inheritance Graphs in Object Database Design , 1996, ER.

[69]  Jens Lehmann,et al.  ORE - A Tool for Repairing and Enriching Knowledge Bases , 2010, SEMWEB.

[70]  Gaël Richard,et al.  Automatic transcription of drum loops , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[71]  Nicolas D. Chetry,et al.  Computer Models for Musical Instrument Identification , 2006 .

[72]  A critical assessment of text mining methods in molecular biology. Proceedings of a workshop. March 28-31, 2004. Granada, Spain. , 2005, BMC bioinformatics.

[73]  Asunción Gómez-Pérez,et al.  The Integration of OntoClean in WebODE , 2002, EON.

[74]  Marc Leman,et al.  Classification of Percussive Sounds using Support Vector Machines , 2004 .

[75]  Dunja Mladenic,et al.  Semi-automatic Construction of Topic Ontologies , 2005, EWMF/KDO.

[76]  Mohammed Bennamoun,et al.  Acquiring Semantic Relations Using the Web for Constructing Lightweight Ontologies , 2009, PAKDD.

[77]  Aaron E. Rosenberg,et al.  Evaluation of a vector quantization talker recognition system in text independent and text dependent modes , 1987 .

[78]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[79]  Bozena Kostek Soft Computing in Acoustics: Applications of Neural Networks, Fuzzy Logic and Rough Sets to Musical Acoustics , 1999 .

[80]  Gert R. G. Lanckriet,et al.  Semantic Annotation and Retrieval of Music and Sound Effects , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[81]  Frank van Harmelen,et al.  A Semantic Web Primer, 2nd Edition (Cooperative Information Systems) , 2008 .

[82]  P. Lachenbruch Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[83]  Marc Ehrig,et al.  Similarity for Ontologies - A Comprehensive Framework , 2005, ECIS.

[84]  Chih-Hsun Chou,et al.  On the Studies of Syllable Segmentation and Improving MFCCs for Automatic Birdsong Recognition , 2008, 2008 IEEE Asia-Pacific Services Computing Conference.

[85]  Sridha Sridharan,et al.  Vector quantization based Gaussian modeling for speaker verification , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[86]  Osman Hegazy,et al.  Applying data mining for ontology building , 2008 .

[87]  Rónán O'Beirne,et al.  The Blackwell Guide to the Philosophy of Computing and Information , 2004 .

[88]  D. Sánchez,et al.  Creating Ontologies from Web documents , 2004 .

[89]  R. Porzel,et al.  A Task-based Approach for Ontology Evaluation , 2022 .

[90]  Hasan Davulcu,et al.  OntoMiner: bootstrapping ontologies from overlapping domain specific web sites , 2004, WWW Alt. '04.

[91]  Tao Li,et al.  Content-based music similarity search and emotion detection , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[92]  Xavier Rodet,et al.  HIERARCHICAL GAUSSIAN TREE WITH INERTIA RATIO MAXIMIZATION FOR THE CLASSIFICATION OF LARGE MUSICAL INSTRUMENT DATABASES , 2003 .

[93]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[94]  Margaret J. Kartomi,et al.  Instruments: Changing Trends , 2001 .

[95]  Paul Buitelaar,et al.  RelExt: A Tool for Relation Extraction from Text in Ontology Extension , 2005, SEMWEB.

[96]  Douglas Turnbull,et al.  Using Artist Similarity to Propagate Semantic Information , 2009, ISMIR.

[97]  Steffen Staab,et al.  OntoEdit: Multifaceted Inferencing for Ontology Engineering , 2003, J. Data Semant..

[98]  György Fazekas,et al.  Publishing Music Similarity Features on the Semantic Web , 2009, ISMIR.

[99]  Anssi Klapuri,et al.  Musical instrument recognition using cepstral coefficients and temporal features , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[100]  Peter J. Danielsen,et al.  LexOnt: A Semi-Automatic Ontology Creation Tool for Programmable Web , 2012, AAAI Spring Symposium: Intelligent Web Services Meet Social Computing.

[101]  Jos de Bruijn,et al.  GenTax: A Generic Methodology for Deriving OWL and RDF-S Ontologies from Hierarchical Classifications, Thesauri, and Inconsistent Taxonomies , 2007, ESWC.

[102]  Kenneth A. Deffenbacher,et al.  Perception and the Senses , 1979 .

[103]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[104]  Martin Hepp Representing the Hierarchy of Industrial Taxonomies in OWL: The gen/tax Approach , 2005 .

[105]  Yorick Wilks,et al.  Data Driven Ontology Evaluation , 2004, LREC.

[106]  Philipp Cimiano,et al.  Ontology learning and population from text - algorithms, evaluation and applications , 2006 .

[107]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[108]  Kent A. Spackman,et al.  Review: Representing Thoughts, Words, and Things in the UMLS , 1998, J. Am. Medical Informatics Assoc..

[109]  Jason Morris On the Role of Ontology in Modern Expert Systems Development , 2006 .

[110]  Brian Matthews,et al.  Semantic Web Technologies , 2005 .

[111]  Gerd Stumme,et al.  Conceptual Knowledge Discovery in Databases Using Formal Concept Analysis Methods , 1998, PKDD.

[112]  Mike E. Davies,et al.  Musical Instrument Identification using LSF and K-means , 2005 .

[113]  C. J. van Rijsbergen,et al.  Information Retrieval , 1979, Encyclopedia of GIS.

[114]  Kristina Lerman,et al.  Constructing folksonomies from user-specified relations on flickr , 2009, WWW '09.

[115]  György Fazekas,et al.  An Overview of Semantic Web Activities in the OMRAS2 Project , 2010 .

[116]  Lie Lu,et al.  Automatic mood detection and tracking of music audio signals , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[117]  R. Kronland-Martinet,et al.  Acoustical Correlates of Timbre and Expressiveness in Clarinet Performance , 2010 .

[118]  N. J. Davies,et al.  Concept Lattices for Knowledge Management , 1999 .

[119]  Antti Eronen,et al.  Automatic musical instrument recognition , 2001 .

[120]  Carole A. Goble,et al.  Learning domain ontologies for semantic Web service descriptions , 2005, J. Web Semant..

[121]  Jeremy Montagu,et al.  A Proposed New Classification System for Musical Instruments , 1971 .

[122]  Leo Obrst,et al.  The Semantic Web: A Guide to the Future of XML, Web Services and Knowledge Management , 2003 .

[123]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[124]  Christian Bizer,et al.  D2R Server - Publishing Relational Databases on the Semantic Web , 2004 .

[125]  Fabien Gouyon,et al.  Automatic Classification of Drum Sounds: A Comparison of Feature Selection Methods and Classification Techniques , 2002, ICMAI.

[126]  Kristoffer Jensen,et al.  Binary Decision Tree Classification of Musical Sounds , 1999, ICMC.

[127]  Mark A. Musen,et al.  The Knowledge Model of Protégé-2000: Combining Interoperability and Flexibility , 2000, EKAW.

[128]  Perry R. Cook,et al.  Towards Automatic Musical Instrument Timbre Recognition , 2010 .

[129]  Mark D. Plumbley,et al.  Birdsong and C4DM: A survey of UK birdsong and machine recognition for music researchers , 2011 .

[130]  Stephan Baumann,et al.  Using natural language input and audio analysis for a human-oriented MIR system , 2002, Second International Conference on Web Delivering of Music, 2002. WEDELMUSIC 2002. Proceedings..

[131]  Emanuele Pollastri,et al.  Musical Instrument Timbres Classification with Spectral Features , 2001, 2001 IEEE Fourth Workshop on Multimedia Signal Processing (Cat. No.01TH8564).

[132]  György Fazekas,et al.  Structural Decomposition of Recorded Vocal Performances and It's Application to Intelligent Audio Editing , 2007 .

[133]  Denny Vrandecic,et al.  Ontology Evaluation , 2009, Handbook on Ontologies.

[134]  Tadeusz Czaszejko,et al.  Automatic Recognition of Isolated Monophonic Musical Instrument Sounds using kNNC , 2005, Journal of Intelligent Information Systems.

[135]  Yin-Wen Chang,et al.  Low-degree Polynomial Mapping of Data for SVM , 2009 .

[136]  M. Levy,et al.  Signal-based Music Searching and Browsing , 2007, 2007 Digest of Technical Papers International Conference on Consumer Electronics.

[137]  George Tzanetakis,et al.  Manipulation, analysis and retrieval systems for audio signals , 2002 .

[138]  Eric D. Scheirer,et al.  Music Content Analysis through Models of Audition , 1998 .

[139]  Gaël Richard,et al.  Musical instrument recognition on solo performances , 2004, 2004 12th European Signal Processing Conference.

[140]  Paul R. Cohen,et al.  The DARPA High-Performance Knowledge Bases Project , 1998, AI Mag..

[141]  M. Richter Classification and Learning of Similarity Measures , 1993 .