Alert button

Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering

May 18, 2023
Heng-Jui Chang, Alexander H. Liu, James Glass

Figure 1 for Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering
Figure 2 for Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering
Figure 3 for Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering
Figure 4 for Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clustering

Share this with someone who'll enjoy it:

Self-supervised speech representation models have succeeded in various tasks, but improving them for content-related problems using unlabeled data is challenging. We propose speaker-invariant clustering (Spin), a novel self-supervised learning method that clusters speech representations and performs swapped prediction between the original and speaker-perturbed utterances. Spin disentangles speaker information and preserves content representations with just 45 minutes of fine-tuning on a single GPU. Spin improves pre-trained networks and outperforms prior methods in speech recognition and acoustic unit discovery.

* Accepted to Interspeech 2023  
View paper onarxiv icon

Share this with someone who'll enjoy it: