Knowledge-based processing of multicomponent signals in a musical application

Abstract We show how conceptual and architectural tools from AI may be incorporated into the solution of a complex signal processing problem in a musical application. Signals in this application have multiple components with possible overlap in the time–frequency plane. The challenge here is to carry out time–frequency analysis in a manner that allows the various components to be separated from each other. Such time–frequency analysis may be performed by using a filterbank in which the analysis filters are adapted in a data-dependent fashion to separately resolve individual signal components. While there exists a numerical solution for performing analysis filter adaptation through the use of a time–frequency energy concentration measure, this solution suffers from the drawback that it is computationally very inefficient. Furthermore, this solution also does not incorporate knowledge about the time–frequency properties of musical signals. We present a novel solution to the filter adaptation problem that while incorporating higher-level knowledge about musical signals also results in at least an order of magnitude computational savings over the numerical solution. Our solution has been designed on the basis of the IPUS architecture that allows the easy integration of musical knowledge with numerical signal processing algorithms. We have also carried out an implementation of our IPUS-based solution within ICP, a C++ platform for building IPUS applications. The performance of our implementation in separating individual components from musical signals containing a mixture of two notes illustrates its potential for use in a variety of applications.