Alert button
Picture for Zhaoheng Ni

Zhaoheng Ni

Alert button

An Empirical Study on the Impact of Positional Encoding in Transformer-based Monaural Speech Enhancement

Jan 18, 2024
Qiquan Zhang, Meng Ge, Hongxu Zhu, Eliathamby Ambikairajah, Qi Song, Zhaoheng Ni, Haizhou Li

Viaarxiv icon

On The Open Prompt Challenge In Conditional Audio Generation

Nov 01, 2023
Ernie Chang, Sidd Srinivasan, Mahi Luthra, Pin-Jie Lin, Varun Nagaraja, Forrest Iandola, Zechun Liu, Zhaoheng Ni, Changsheng Zhao, Yangyang Shi, Vikas Chandra

Viaarxiv icon

TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch

Oct 27, 2023
Jeff Hwang, Moto Hira, Caroline Chen, Xiaohui Zhang, Zhaoheng Ni, Guangzhi Sun, Pingchuan Ma, Ruizhe Huang, Vineel Pratap, Yuekai Zhang, Anurag Kumar, Chin-Yun Yu, Chuang Zhu, Chunxi Liu, Jacob Kahn, Mirco Ravanelli, Peng Sun, Shinji Watanabe, Yangyang Shi, Yumeng Tao, Robin Scheibler, Samuele Cornell, Sean Kim, Stavros Petridis

Figure 1 for TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch
Figure 2 for TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch
Figure 3 for TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch
Figure 4 for TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch
Viaarxiv icon

Exploring Speech Enhancement for Low-resource Speech Synthesis

Sep 19, 2023
Zhaoheng Ni, Sravya Popuri, Ning Dong, Kohei Saijo, Xiaohui Zhang, Gael Le Lan, Yangyang Shi, Vikas Chandra, Changhan Wang

Figure 1 for Exploring Speech Enhancement for Low-resource Speech Synthesis
Figure 2 for Exploring Speech Enhancement for Low-resource Speech Synthesis
Figure 3 for Exploring Speech Enhancement for Low-resource Speech Synthesis
Figure 4 for Exploring Speech Enhancement for Low-resource Speech Synthesis
Viaarxiv icon

FoleyGen: Visually-Guided Audio Generation

Sep 19, 2023
Xinhao Mei, Varun Nagaraja, Gael Le Lan, Zhaoheng Ni, Ernie Chang, Yangyang Shi, Vikas Chandra

Figure 1 for FoleyGen: Visually-Guided Audio Generation
Figure 2 for FoleyGen: Visually-Guided Audio Generation
Figure 3 for FoleyGen: Visually-Guided Audio Generation
Figure 4 for FoleyGen: Visually-Guided Audio Generation
Viaarxiv icon

Stack-and-Delay: a new codebook pattern for music generation

Sep 15, 2023
Gael Le Lan, Varun Nagaraja, Ernie Chang, David Kant, Zhaoheng Ni, Yangyang Shi, Forrest Iandola, Vikas Chandra

Figure 1 for Stack-and-Delay: a new codebook pattern for music generation
Figure 2 for Stack-and-Delay: a new codebook pattern for music generation
Figure 3 for Stack-and-Delay: a new codebook pattern for music generation
Figure 4 for Stack-and-Delay: a new codebook pattern for music generation
Viaarxiv icon

Enhance audio generation controllability through representation similarity regularization

Sep 15, 2023
Yangyang Shi, Gael Le Lan, Varun Nagaraja, Zhaoheng Ni, Xinhao Mei, Ernie Chang, Forrest Iandola, Yang Liu, Vikas Chandra

Figure 1 for Enhance audio generation controllability through representation similarity regularization
Figure 2 for Enhance audio generation controllability through representation similarity regularization
Figure 3 for Enhance audio generation controllability through representation similarity regularization
Figure 4 for Enhance audio generation controllability through representation similarity regularization
Viaarxiv icon

Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Compute

Jun 11, 2023
William Chen, Xuankai Chang, Yifan Peng, Zhaoheng Ni, Soumi Maiti, Shinji Watanabe

Figure 1 for Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Compute
Figure 2 for Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Compute
Figure 3 for Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Compute
Figure 4 for Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Compute
Viaarxiv icon

Scaling Speech Technology to 1,000+ Languages

May 22, 2023
Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi, Alexei Baevski, Yossi Adi, Xiaohui Zhang, Wei-Ning Hsu, Alexis Conneau, Michael Auli

Figure 1 for Scaling Speech Technology to 1,000+ Languages
Figure 2 for Scaling Speech Technology to 1,000+ Languages
Figure 3 for Scaling Speech Technology to 1,000+ Languages
Figure 4 for Scaling Speech Technology to 1,000+ Languages
Viaarxiv icon

Ripple sparse self-attention for monaural speech enhancement

May 15, 2023
Qiquan Zhang, Hongxu Zhu, Qi Song, Xinyuan Qian, Zhaoheng Ni, Haizhou Li

Figure 1 for Ripple sparse self-attention for monaural speech enhancement
Figure 2 for Ripple sparse self-attention for monaural speech enhancement
Figure 3 for Ripple sparse self-attention for monaural speech enhancement
Figure 4 for Ripple sparse self-attention for monaural speech enhancement
Viaarxiv icon