IUT Informatique

Spectral models provide general representations for sound well-suited for expressive musical transformations. These models allow us to extract and modify perceptually-relevant parameters such as amplitude, frequency, and spectrum. Thus, they are of great interest for the classification of musical sounds. A new analysis method was proposed to accurately extract the spectral parameters for the model from existing sounds. This method extends the classic short-time Fourier analysis by also considering the derivatives of the sound signal, and it can work with very short analysis windows. Although originally designed for stationary sounds with no noise, this method shows excellent results in the presence of noise and it is currently being extended in order to handle nonstationary sounds as well. A very efficient synthesis algorithm, based on a recursive description of the sine function, is able to reproduce sound in real time from the model parameters. This algorithm allows an extremely fine control of the partials of the sounds while avoiding signal discontinuities as well as numerical imprecision, and with a nearly optimal number of operations per partial. Psychoacoustic phenomena such as masking are considered in order to reduce on the fly the number of partials to be synthesized.

ABSTRACT Spectral models attempt to parametrize sound at the basilar membrane of the ear, thus permitting transformations closely linked to the perception. However, for high-quality real-time applications, these models require methods for a precise analysis and an efficient synthesis. When dealing with musical sound, that is, a polyphonic mix of non-stationary complex sounds, the main challenge is to extract these different sounds present in the musical mix. This can be done using (psycho) acoustical knowledge about the sound sources (computational auditory scene analysis approach). However, the quality is often not sufficient. When access to the compositional process is given, another option is to use some bits of this ground truth as additional information (informed analysis approach). This more precise analysis allows deeper transformations, and using psycho-acoustical considerations, efficient data structures, and algorithms, it is possible to re-synthesize the sounds from the model parameters in real time and with a high quality. This opens up new impressive applications, such as "active listening", enabling the listener to interact with the sound while it is played. The musical parameters (e.g., loudness or spatial location) of the sound sources present in the musical mix can now be changed interactively.

Sinusoidal modeling has been successfully applied to a wide range of audio signal processing problems, such as coding or time and frequency stretching. While many methods have been proposed for the analysis part of the process, it seems that there is some general agreement concerning the synthesis part in the nonoverlapping case: It is very often achieved by using the well-known McAulay-Quatieri method, which consists of an order 3 polynomial reconstruction of the phases of the sinusoidal model partials. We compare this "classical" approach with both a simpler (order 1, that is, linear interpolation) and a more complex (order 5) polynomial model for phase interpolation of quasi-harmonic signals. A gain has been measured in the signal-to-noise ratio at the synthesis stage, although the performance is limited by the amplitude model and by the imprecision in the analysis stage.

Abstract: In this paper, we introduce a new partial tracking method suitablefor the sinusoidal modeling of mixtures of instrumental soundswith pseudo-stationary frequencies. This method, based on the linearprediction of the frequency evolutions of the partials, enablesus to track these partials more accurately at the analysis stage, evenin complex sound mixtures. This allows our spectral model to betterhandle polyphonic sounds.

ABSTRACT In this work we readdress the issue of audio source separation in an informed scenario, where certain information about the sound sources is embedded into their mixture as an imperceptible watermark. In doing so, we provide a description of an improved algorithm that follows the linearly constrained minimum-variance filtering approach in the subband domain, in order to obtain perceptually better estimates of the source signals in comparison to other published approaches. Just as its predecessor, the algorithm does not impose any restrictions on the number of simultaneously active sources, neither on their spectral overlap. It rather adapts to a given signal constellation and provides the best possible estimates under given constraints in linearithmic time. The validity of the approach is demonstrated on a stereo mixture with two levels of sound complexity. It is also shown by means of both objective and subjective evaluation that the proposed algorithm outperforms a reference algorithm by at least one grade. Bearing high perceptual resemblance to the original signals at a fairly tolerable data rate of 10-20 kbps per source, the algorithm hence seems well-suited for active listening applications such as re-mixing or re-spatialization in real time.