Alert button

Building Robust Spoken Language Understanding by Cross Attention between Phoneme Sequence and ASR Hypothesis

Mar 22, 2022
Zexun Wang, Yuquan Le, Yi Zhu, Yuming Zhao, Mingchao Feng, Meng Chen, Xiaodong He

Figure 1 for Building Robust Spoken Language Understanding by Cross Attention between Phoneme Sequence and ASR Hypothesis
Figure 2 for Building Robust Spoken Language Understanding by Cross Attention between Phoneme Sequence and ASR Hypothesis
Figure 3 for Building Robust Spoken Language Understanding by Cross Attention between Phoneme Sequence and ASR Hypothesis
Figure 4 for Building Robust Spoken Language Understanding by Cross Attention between Phoneme Sequence and ASR Hypothesis

Share this with someone who'll enjoy it:

Building Spoken Language Understanding (SLU) robust to Automatic Speech Recognition (ASR) errors is an essential issue for various voice-enabled virtual assistants. Considering that most ASR errors are caused by phonetic confusion between similar-sounding expressions, intuitively, leveraging the phoneme sequence of speech can complement ASR hypothesis and enhance the robustness of SLU. This paper proposes a novel model with Cross Attention for SLU (denoted as CASLU). The cross attention block is devised to catch the fine-grained interactions between phoneme and word embeddings in order to make the joint representations catch the phonetic and semantic features of input simultaneously and for overcoming the ASR errors in downstream natural language understanding (NLU) tasks. Extensive experiments are conducted on three datasets, showing the effectiveness and competitiveness of our approach. Additionally, We also validate the universality of CASLU and prove its complementarity when combining with other robust SLU techniques.

* ICASSP 2022  
View paper onarxiv icon

Share this with someone who'll enjoy it: