Deep clustering: Discriminative embeddings for segmentation and separation
Citations Over TimeTop 1% of 2016 papers
Abstract
We address the problem of "cocktail-party" source separation in a deep learning framework called deep clustering. Previous deep network approaches to separation have shown promising performance in scenarios with a fixed number of sources, each belonging to a distinct signal class, such as speech and noise. However, for arbitrary source classes and number, "class-based" methods are not suitable. Instead, we train a deep network to assign contrastive embedding vectors to each time-frequency region of the spectrogram in order to implicitly predict the segmentation labels of the target spectrogram from the input mixtures. This yields a deep network-based analogue to spectral clustering, in that the embeddings form a low-rank pair-wise affinity matrix that approximates the ideal affinity matrix, while enabling much faster performance. At test time, the clustering step "decodes" the segmentation implicit in the embeddings by optimizing K-means with respect to the unknown assignments. Preliminary experiments on single-channel mixtures from multiple speakers show that a speaker-independent model trained on two-speaker mixtures can improve signal quality for mixtures of held-out speakers by an average of 6dB. More dramatically, the same model does surprisingly well with three-speaker mixtures.
Related Papers
- → SW-WAVENET: Learning Representation from Spectrogram and Wavegram Using Wavenet for Anomalous Sound Detection(2023)19 cited
- → Using the reassigned spectrogram to obtain a voiceprint(2006)
- → The preliminary application of Gabor spectrogram analysis in speech samples(1993)
- Estimation of Clean Spectrogram Noisy Value Functions Based on Metropolis Iterative Algorithm.(2013)
- → Anatomy of a Spectrogram(2024)