Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Jonathan Le Roux

Jonathan Le Roux

The Cocktail Fork Problem: Three-Stem Audio Separation for Real-World Soundtracks


Oct 19, 2021
Darius Petermann, Gordon Wichern, Zhong-Qiu Wang, Jonathan Le Roux

Figure 1 for The Cocktail Fork Problem: Three-Stem Audio Separation for Real-World Soundtracks
Figure 2 for The Cocktail Fork Problem: Three-Stem Audio Separation for Real-World Soundtracks
Figure 3 for The Cocktail Fork Problem: Three-Stem Audio Separation for Real-World Soundtracks
Figure 4 for The Cocktail Fork Problem: Three-Stem Audio Separation for Real-World Soundtracks
* Submitted to ICASSP2022. For resources and examples, see https://cocktail-fork.github.io 
  Access Paper or Ask Questions

Audio-Visual Scene-Aware Dialog and Reasoning using Audio-Visual Transformers with Joint Student-Teacher Learning


Oct 13, 2021
Ankit P. Shah, Shijie Geng, Peng Gao, Anoop Cherian, Takaaki Hori, Tim K. Marks, Jonathan Le Roux, Chiori Hori

Figure 1 for Audio-Visual Scene-Aware Dialog and Reasoning using Audio-Visual Transformers with Joint Student-Teacher Learning
Figure 2 for Audio-Visual Scene-Aware Dialog and Reasoning using Audio-Visual Transformers with Joint Student-Teacher Learning
Figure 3 for Audio-Visual Scene-Aware Dialog and Reasoning using Audio-Visual Transformers with Joint Student-Teacher Learning
Figure 4 for Audio-Visual Scene-Aware Dialog and Reasoning using Audio-Visual Transformers with Joint Student-Teacher Learning
* https://dstc10.dstc.community/home and https://github.com/dialogtekgeek/AVSD-DSTC10_Official/ 
  Access Paper or Ask Questions

Advancing Momentum Pseudo-Labeling with Conformer and Initialization Strategy


Oct 11, 2021
Yosuke Higuchi, Niko Moritz, Jonathan Le Roux, Takaaki Hori

Figure 1 for Advancing Momentum Pseudo-Labeling with Conformer and Initialization Strategy
Figure 2 for Advancing Momentum Pseudo-Labeling with Conformer and Initialization Strategy
Figure 3 for Advancing Momentum Pseudo-Labeling with Conformer and Initialization Strategy
* Submitted to ICASSP2022 
  Access Paper or Ask Questions

Leveraging Low-Distortion Target Estimates for Improved Speech Enhancement


Oct 01, 2021
Zhong-Qiu Wang, Gordon Wichern, Jonathan Le Roux

Figure 1 for Leveraging Low-Distortion Target Estimates for Improved Speech Enhancement
Figure 2 for Leveraging Low-Distortion Target Estimates for Improved Speech Enhancement
Figure 3 for Leveraging Low-Distortion Target Estimates for Improved Speech Enhancement
Figure 4 for Leveraging Low-Distortion Target Estimates for Improved Speech Enhancement
* in submission 
  Access Paper or Ask Questions

Visual Scene Graphs for Audio Source Separation


Sep 24, 2021
Moitreya Chatterjee, Jonathan Le Roux, Narendra Ahuja, Anoop Cherian

Figure 1 for Visual Scene Graphs for Audio Source Separation
Figure 2 for Visual Scene Graphs for Audio Source Separation
Figure 3 for Visual Scene Graphs for Audio Source Separation
Figure 4 for Visual Scene Graphs for Audio Source Separation
* Accepted at ICCV 2021 
  Access Paper or Ask Questions

Convolutive Prediction for Monaural Speech Dereverberation and Noisy-Reverberant Speaker Separation


Aug 16, 2021
Zhong-Qiu Wang, Gordon Wichern, Jonathan Le Roux

Figure 1 for Convolutive Prediction for Monaural Speech Dereverberation and Noisy-Reverberant Speaker Separation
Figure 2 for Convolutive Prediction for Monaural Speech Dereverberation and Noisy-Reverberant Speaker Separation
Figure 3 for Convolutive Prediction for Monaural Speech Dereverberation and Noisy-Reverberant Speaker Separation
Figure 4 for Convolutive Prediction for Monaural Speech Dereverberation and Noisy-Reverberant Speaker Separation
* 16 pages, 4 figures 
  Access Paper or Ask Questions

Convolutive Prediction for Reverberant Speech Separation


Aug 16, 2021
Zhong-Qiu Wang, Gordon Wichern, Jonathan Le Roux

Figure 1 for Convolutive Prediction for Reverberant Speech Separation
Figure 2 for Convolutive Prediction for Reverberant Speech Separation
Figure 3 for Convolutive Prediction for Reverberant Speech Separation
* in IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2021 
  Access Paper or Ask Questions

On The Compensation Between Magnitude and Phase in Speech Separation


Aug 11, 2021
Zhong-Qiu Wang, Gordon Wichern, Jonathan Le Roux

Figure 1 for On The Compensation Between Magnitude and Phase in Speech Separation
Figure 2 for On The Compensation Between Magnitude and Phase in Speech Separation
Figure 3 for On The Compensation Between Magnitude and Phase in Speech Separation
Figure 4 for On The Compensation Between Magnitude and Phase in Speech Separation
* in submission to IEEE Signal Processing Letters 
  Access Paper or Ask Questions

Optimizing Latency for Online Video CaptioningUsing Audio-Visual Transformers


Aug 04, 2021
Chiori Hori, Takaaki Hori, Jonathan Le Roux

Figure 1 for Optimizing Latency for Online Video CaptioningUsing Audio-Visual Transformers
Figure 2 for Optimizing Latency for Online Video CaptioningUsing Audio-Visual Transformers
Figure 3 for Optimizing Latency for Online Video CaptioningUsing Audio-Visual Transformers
Figure 4 for Optimizing Latency for Online Video CaptioningUsing Audio-Visual Transformers
* Interspeech 2021 accepted 
  Access Paper or Ask Questions

Dual Causal/Non-Causal Self-Attention for Streaming End-to-End Speech Recognition


Jul 02, 2021
Niko Moritz, Takaaki Hori, Jonathan Le Roux

Figure 1 for Dual Causal/Non-Causal Self-Attention for Streaming End-to-End Speech Recognition
Figure 2 for Dual Causal/Non-Causal Self-Attention for Streaming End-to-End Speech Recognition
* Accepted to Interspeech 2021 
  Access Paper or Ask Questions

Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition


Jun 16, 2021
Yosuke Higuchi, Niko Moritz, Jonathan Le Roux, Takaaki Hori

Figure 1 for Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition
Figure 2 for Momentum Pseudo-Labeling for Semi-Supervised Speech Recognition
* Accepted to Interspeech 2021 
  Access Paper or Ask Questions

Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformers


Apr 19, 2021
Takaaki Hori, Niko Moritz, Chiori Hori, Jonathan Le Roux

Figure 1 for Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformers
Figure 2 for Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformers
Figure 3 for Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformers
* Submitted to INTERSPEECH 2021 
  Access Paper or Ask Questions

Capturing Multi-Resolution Context by Dilated Self-Attention


Apr 07, 2021
Niko Moritz, Takaaki Hori, Jonathan Le Roux

Figure 1 for Capturing Multi-Resolution Context by Dilated Self-Attention
Figure 2 for Capturing Multi-Resolution Context by Dilated Self-Attention
Figure 3 for Capturing Multi-Resolution Context by Dilated Self-Attention
* In Proc. ICASSP 2021 
  Access Paper or Ask Questions

Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Training


Nov 26, 2020
Sameer Khurana, Niko Moritz, Takaaki Hori, Jonathan Le Roux

Figure 1 for Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Training
Figure 2 for Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Training
Figure 3 for Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Training
Figure 4 for Unsupervised Domain Adaptation for Speech Recognition via Uncertainty Driven Self-Training
* Submitted to ICASSP 2021 
  Access Paper or Ask Questions

Semi-Supervised Speech Recognition via Graph-based Temporal Classification


Oct 29, 2020
Niko Moritz, Takaaki Hori, Jonathan Le Roux

Figure 1 for Semi-Supervised Speech Recognition via Graph-based Temporal Classification
Figure 2 for Semi-Supervised Speech Recognition via Graph-based Temporal Classification
Figure 3 for Semi-Supervised Speech Recognition via Graph-based Temporal Classification
Figure 4 for Semi-Supervised Speech Recognition via Graph-based Temporal Classification
* Submitted to ICASSP 2021 
  Access Paper or Ask Questions

Transcription Is All You Need: Learning to Separate Musical Mixtures with Score as Supervision


Oct 22, 2020
Yun-Ning Hung, Gordon Wichern, Jonathan Le Roux

Figure 1 for Transcription Is All You Need: Learning to Separate Musical Mixtures with Score as Supervision
Figure 2 for Transcription Is All You Need: Learning to Separate Musical Mixtures with Score as Supervision
Figure 3 for Transcription Is All You Need: Learning to Separate Musical Mixtures with Score as Supervision
Figure 4 for Transcription Is All You Need: Learning to Separate Musical Mixtures with Score as Supervision

  Access Paper or Ask Questions

Multi-Pass Transformer for Machine Translation


Sep 23, 2020
Peng Gao, Chiori Hori, Shijie Geng, Takaaki Hori, Jonathan Le Roux

Figure 1 for Multi-Pass Transformer for Machine Translation
Figure 2 for Multi-Pass Transformer for Machine Translation
Figure 3 for Multi-Pass Transformer for Machine Translation
Figure 4 for Multi-Pass Transformer for Machine Translation
* 10 pages, 5 figures and 2 tables 
  Access Paper or Ask Questions

AutoClip: Adaptive Gradient Clipping for Source Separation Networks


Jul 25, 2020
Prem Seetharaman, Gordon Wichern, Bryan Pardo, Jonathan Le Roux

Figure 1 for AutoClip: Adaptive Gradient Clipping for Source Separation Networks
Figure 2 for AutoClip: Adaptive Gradient Clipping for Source Separation Networks
* Accepted at 2020 IEEE International Workshop on Machine Learning for Signal Processing, Sept.\ 21--24, 2020, Espoo, Finland 
  Access Paper or Ask Questions

Spatio-Temporal Scene Graphs for Video Dialog


Jul 08, 2020
Shijie Geng, Peng Gao, Chiori Hori, Jonathan Le Roux, Anoop Cherian

Figure 1 for Spatio-Temporal Scene Graphs for Video Dialog
Figure 2 for Spatio-Temporal Scene Graphs for Video Dialog
Figure 3 for Spatio-Temporal Scene Graphs for Video Dialog
Figure 4 for Spatio-Temporal Scene Graphs for Video Dialog

  Access Paper or Ask Questions

Detecting Audio Attacks on ASR Systems with Dropout Uncertainty


Jun 02, 2020
Tejas Jayashankar, Jonathan Le Roux, Pierre Moulin

Figure 1 for Detecting Audio Attacks on ASR Systems with Dropout Uncertainty
Figure 2 for Detecting Audio Attacks on ASR Systems with Dropout Uncertainty

  Access Paper or Ask Questions

Unsupervised Speaker Adaptation using Attention-based Speaker Memory for End-to-End ASR


Feb 14, 2020
Leda SarΔ±, Niko Moritz, Takaaki Hori, Jonathan Le Roux

Figure 1 for Unsupervised Speaker Adaptation using Attention-based Speaker Memory for End-to-End ASR
Figure 2 for Unsupervised Speaker Adaptation using Attention-based Speaker Memory for End-to-End ASR
Figure 3 for Unsupervised Speaker Adaptation using Attention-based Speaker Memory for End-to-End ASR
Figure 4 for Unsupervised Speaker Adaptation using Attention-based Speaker Memory for End-to-End ASR
* To appear in Proc. ICASSP 2020 
  Access Paper or Ask Questions

End-to-End Multi-speaker Speech Recognition with Transformer


Feb 13, 2020
Xuankai Chang, Wangyou Zhang, Yanmin Qian, Jonathan Le Roux, Shinji Watanabe

Figure 1 for End-to-End Multi-speaker Speech Recognition with Transformer
Figure 2 for End-to-End Multi-speaker Speech Recognition with Transformer
Figure 3 for End-to-End Multi-speaker Speech Recognition with Transformer
Figure 4 for End-to-End Multi-speaker Speech Recognition with Transformer
* To appear in ICASSP 2020 
  Access Paper or Ask Questions

Streaming automatic speech recognition with the transformer model


Jan 09, 2020
Niko Moritz, Takaaki Hori, Jonathan Le Roux

Figure 1 for Streaming automatic speech recognition with the transformer model
Figure 2 for Streaming automatic speech recognition with the transformer model
Figure 3 for Streaming automatic speech recognition with the transformer model

  Access Paper or Ask Questions

Finding Strength in Weakness: Learning to Separate Sounds with Weak Supervision


Nov 06, 2019
Fatemeh Pishdadian, Gordon Wichern, Jonathan Le Roux

Figure 1 for Finding Strength in Weakness: Learning to Separate Sounds with Weak Supervision
Figure 2 for Finding Strength in Weakness: Learning to Separate Sounds with Weak Supervision
Figure 3 for Finding Strength in Weakness: Learning to Separate Sounds with Weak Supervision
Figure 4 for Finding Strength in Weakness: Learning to Separate Sounds with Weak Supervision

  Access Paper or Ask Questions

Bootstrapping deep music separation from primitive auditory grouping principles


Oct 23, 2019
Prem Seetharaman, Gordon Wichern, Jonathan Le Roux, Bryan Pardo

Figure 1 for Bootstrapping deep music separation from primitive auditory grouping principles
Figure 2 for Bootstrapping deep music separation from primitive auditory grouping principles
Figure 3 for Bootstrapping deep music separation from primitive auditory grouping principles
Figure 4 for Bootstrapping deep music separation from primitive auditory grouping principles

  Access Paper or Ask Questions

MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition


Oct 16, 2019
Xuankai Chang, Wangyou Zhang, Yanmin Qian, Jonathan Le Roux, Shinji Watanabe

Figure 1 for MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition
Figure 2 for MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition
Figure 3 for MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition
Figure 4 for MIMO-SPEECH: End-to-End Multi-Channel Multi-Speaker Speech Recognition
* Accepted at ASRU 2019 
  Access Paper or Ask Questions

Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantity


Sep 18, 2019
Ethan Manilow, Gordon Wichern, Prem Seetharaman, Jonathan Le Roux

Figure 1 for Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantity
Figure 2 for Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantity
Figure 3 for Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantity
Figure 4 for Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantity
* Accepted for publication at WASPAA 2019 
  Access Paper or Ask Questions

WHAM!: Extending Speech Separation to Noisy Environments


Jul 02, 2019
Gordon Wichern, Joe Antognini, Michael Flynn, Licheng Richard Zhu, Emmett McQuinn, Dwight Crow, Ethan Manilow, Jonathan Le Roux

Figure 1 for WHAM!: Extending Speech Separation to Noisy Environments
Figure 2 for WHAM!: Extending Speech Separation to Noisy Environments
Figure 3 for WHAM!: Extending Speech Separation to Noisy Environments
Figure 4 for WHAM!: Extending Speech Separation to Noisy Environments
* Accepted for publication at Interspeech 2019 
  Access Paper or Ask Questions

Universal Sound Separation


May 08, 2019
Ilya Kavalerov, Scott Wisdom, Hakan Erdogan, Brian Patton, Kevin Wilson, Jonathan Le Roux, John R. Hershey

Figure 1 for Universal Sound Separation
Figure 2 for Universal Sound Separation
Figure 3 for Universal Sound Separation
Figure 4 for Universal Sound Separation
* 5 pages, submitted to WASPAA 2019 
  Access Paper or Ask Questions