Alert button

Self-supervised Learning with Speech Modulation Dropout

Mar 22, 2023
Samik Sadhu, Hynek Hermansky

Figure 1 for Self-supervised Learning with Speech Modulation Dropout
Figure 2 for Self-supervised Learning with Speech Modulation Dropout
Figure 3 for Self-supervised Learning with Speech Modulation Dropout
Figure 4 for Self-supervised Learning with Speech Modulation Dropout

Share this with someone who'll enjoy it:

We show that training a multi-headed self-attention-based deep network to predict deleted, information-dense 2-8 Hz speech modulations over a 1.5-second section of a speech utterance is an effective way to make machines learn to extract speech modulations using time-domain contextual information. Our work exhibits that, once trained on large volumes of unlabelled data, the outputs of the self-attention layers vary in time with a modulation peak at 4 Hz. These pre-trained layers can be used to initialize parts of an Automatic Speech Recognition system to reduce its reliance on labeled speech data greatly.

View paper onarxiv icon

Share this with someone who'll enjoy it: