TarsosDSP, a Real-Time Audio Processing Framework in Java

This paper presents TarsosDSP, a framework for real-time audio analysis and processing. Most libraries and frameworks offer either audio analysis and feature extraction or audio synthesis and processing. TarsosDSP is one of a only a few frameworks that offers both analysis, processing and feature extraction in real-time, a unique feature in the Java ecosystem. The framework contains practical audio processing algorithms, it can be extended easily, and has no external dependencies. Each algorithm is implemented as simple as possible thanks to a straightforward processing pipeline. TarsosDSP's features include a resampling algorithm, onset detectors, a number of pitch estimation algorithms, a time stretch algorithm, a pitch shifting algorithm, and an algorithm to calculate the Constant-Q. The framework also allows simple audio synthesis, some audio effects, and several filters. The Open Source framework is a valuable contribution to the MIR-Community and ideal fit for interactive MIR-applications on Android.

[1]  M. Ross,et al.  Average magnitude difference function pitch extractor , 1974 .

[2]  Pau Arumí,et al.  CLAM, yet another library for audio and music processing? , 2002, OOPSLA '02.

[3]  Perry R. Cook,et al.  Rtmidi, Rtaudio, and a synthesis Toolkit (STK) Update , 2005, ICMC.

[4]  Santiago David Davila Benavides.  Raciocínio de agentes musicais composição algorítmica, vida artificial e interatividade em sistemas multiagentes musicais , 2012 .

[5]  Peter Kulchyski.  and , 2015 .

[6]  Dan Barry,et al.  Drum Source Separation using Percussive Feature Detection and Spectral Modulation , 2005 .

[7]  Mark B. Sandler,et al.  The Sonic Visualiser: A Visualisation Platform for Semantic Descriptors from Musical Signals , 2006, ISMIR.

[8]  Simon Dixon,et al.  Automatic Extraction of Tempo and Beat From Expressive Performances , 2001 .

[9]  Marc Leman,et al.  Tarsos, a Modular Platform for Precise Pitch Analysis of Western and Non-Western Music , 2013 .

[10]  Phil Burk.  JSyn - A Real-time Synthesis API for Java , 1998, ICMC.

[11]  Eric Larson,et al.  Real-Time Time-Domain Pitch Tracking Using Wavelets , 2006 .

[12]  Xavier Serra,et al.  Essentia: An Audio Analysis Library for Music Information Retrieval , 2013, ISMIR.

[13]  George Tzanetakis,et al.  MARSYAS: a framework for audio analysis , 1999, Organised Sound.

[14]  Ichiro Fujinaga,et al.  jAudio: An Feature Extraction Library , 2005, ISMIR.

[15]  Geoff Wyvill,et al.  A Smarter Way to Find pitch , 2005, ICMC.

[16]  Julius O. Smith,et al.  A flexible sampling-rate conversion method , 1984, ICASSP.

[17]  Hideki Kawahara,et al.  YIN, a fundamental frequency estimator for speech and music. , 2002, The Journal of the Acoustical Society of America.

[18]  Judith C. Brown,et al.  An efficient algorithm for the calculation of a constant Q transform , 1992 .

[19]  Chandrasekhar Ramakrishnan,et al.  The CREATE Signal Library ("Sizzle"): Design, Issues, and Applications , 2003, ICMC.

[20]  Xavier Amatriain,et al.  An Object-oriented metamodel for digital signal processing with a focus on audio and music , 2005 .

[21]  Stefan Müller Arisona,et al.  Real-Time, low Latency audio Processing in Java , 2007, ICMC.

[22]  Philip McLeod,et al.  Fast, Accurate Pitch Detection Tools for Music Analysis , 2008 .

[23]  Dinesh K. Pai,et al.  JASS: A JAVA AUDIO SYNTHESIS SYSTEM FOR PROGRAMMERS , 2001 .

[24]  Victor Lazzarini.  SOUND PROCESSING WITH THE SndObj LIBRARY: AN OVERVIEW , 2001 .

[25]  Werner Verhelst,et al.  An overlap-add technique based on waveform similarity (WSOLA) for high quality time-scale modification of speech , 1993, 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[26]  Paul M. Brossier,et al.  Automatic annotation of musical audio for interactive applications , 2006 .

[27]  John A. Mills,et al.  Music Programming in Minim , 2010, NIME.

[28]  Thomas Fillon,et al.  YAAFE, an Easy to Use and Efficient Audio Feature Extraction Software , 2010, ISMIR.