Statistical graphical models for scene analysis, source separation and other audio applications

Statistical Graphical Models for Scene Analysis, Source Separation and Other Audio Applications Manuel J. Reyes Gómez The problem of separating overlapping sound sources has long been a research goal in sound processing, not least because of the apparent ease with which we as listeners achieve perceptual separation and isolation of sound sources in our everyday experiences. Human listeners use their prior knowledge of all the sound classes that they have experienced through their lives to impose constraints on the form that elements on a mixture can take. Listeners use the information obtained from partial observation of the unmixed context to disambiguate the components where the energy is locally swamped by interfering sources. Researchers working on this problem (Ellis 1996) argue that just as human listeners have top-down knowledge, prior constraints on the form that the mixture components can take is the critical component to making source separation systems work. In this thesis, we propose to encode these contraints in the form of models which capture the statistical distributions of the features of mixture components, using the framework of statistical graphical models, and then use those models to estimate obscured or corrupted portions from partial observations. Our overarching goal is to explain composed data as a composition of the models of the individual sources. After reviewing the basic statistical tools, this dissertation describes three models of this kind. The first uses multiple-microphone recordings from reverberant rooms combined in a filter-and-sum setup. The filter coefficients are optimized to match system output against a model of speech taken from a speech recognizer. The second model addresses the more difficult case of a single channel recording, and handles the tractability problems of the very large number of states required by decomposing the signal into subbands. The final model provides very precise fits to source signals without an enormous dictionary of prototypes, but instead by exploiting the observation that much of a real-world signal can be described as systematic local spectral deformations of adjacent time frames; by inferring these deformations between occasional spectral templates, the entire sound is accurately described. For this last model, we show in detail how a mixture of two sources can be segmented at points where local deformations do not provide adequate explanation, to delineate regions dominated by one source. Individual sources can then be reconstructed by interpolation of the deformation parameters to reconstruct estimates of the mixture components even when they are hidden behind high-energy maskers. Although acoustic scene analysis and source separation are used as motivating and illustrative applications throught, the intrinsic descriptions of the nature of sound sources captured by these models could have other, broader applications in signal recognition, compression and modification, and even beyond audio in other domains where signal properties have the appropriate nontrivial local structure.