Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation

May 11, 2020
Aditya Siddhant, Ankur Bapna, Yuan Cao, Orhan Firat, Mia Chen, Sneha Kudugunta, Naveen Arivazhagan, Yonghui Wu

* ACL 2020 

  Access Model/Code and Paper
Your GAN is Secretly an Energy-based Model and You Should use Discriminator Driven Latent Sampling

Mar 24, 2020
Tong Che, Ruixiang Zhang, Jascha Sohl-Dickstein, Hugo Larochelle, Liam Paull, Yuan Cao, Yoshua Bengio


  Access Model/Code and Paper
Echo State Neural Machine Translation

Feb 27, 2020
Ankush Garg, Yuan Cao, Qi Ge


  Access Model/Code and Paper
Mean-Field Analysis of Two-Layer Neural Networks: Non-Asymptotic Rates and Generalization Bounds

Feb 10, 2020
Zixiang Chen, Yuan Cao, Quanquan Gu, Tong Zhang

* 50 pages, 1 table 

  Access Model/Code and Paper
Fully-hierarchical fine-grained prosody modeling for interpretable speech synthesis

Feb 06, 2020
Guangzhi Sun, Yu Zhang, Ron J. Weiss, Yuan Cao, Heiga Zen, Yonghui Wu

* to appear in ICASSP 2020 

  Access Model/Code and Paper
Generating diverse and natural text-to-speech samples using a quantized fine-grained VAE and auto-regressive prosody prior

Feb 06, 2020
Guangzhi Sun, Yu Zhang, Ron J. Weiss, Yuan Cao, Heiga Zen, Andrew Rosenberg, Bhuvana Ramabhadran, Yonghui Wu

* To appear in ICASSP 2020 

  Access Model/Code and Paper
Towards Understanding the Spectral Bias of Deep Learning

Dec 03, 2019
Yuan Cao, Zhiying Fang, Yue Wu, Ding-Xuan Zhou, Quanquan Gu

* 26 pages, 4 figures 

  Access Model/Code and Paper
How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?

Nov 27, 2019
Zixiang Chen, Yuan Cao, Difan Zou, Quanquan Gu

* 27 pages 

  Access Model/Code and Paper
Tight Sample Complexity of Learning One-hidden-layer Convolutional Neural Networks

Nov 12, 2019
Yuan Cao, Quanquan Gu

* 45 pages, 3 figures, 1 table. In NeurIPS 2019 

  Access Model/Code and Paper
Algorithm-Dependent Generalization Bounds for Overparameterized Deep Residual Networks

Oct 07, 2019
Spencer Frei, Yuan Cao, Quanquan Gu

* 37 pages. In NeurIPS 2019 

  Access Model/Code and Paper
Video Prediction for Precipitation Nowcasting

Jul 18, 2019
Yuan Cao, Qiuying Li, Lei Chen, Junping Zhang, Leiming Ma

* 13 pages, 7 figures, the-state-of-the-art on Monving-MNIST Dataset 

  Access Model/Code and Paper
Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges

Jul 11, 2019
Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Dmitry Lepikhin, Melvin Johnson, Maxim Krikun, Mia Xu Chen, Yuan Cao, George Foster, Colin Cherry, Wolfgang Macherey, Zhifeng Chen, Yonghui Wu


  Access Model/Code and Paper
Neural Decipherment via Minimum-Cost Flow: from Ugaritic to Linear B

Jun 16, 2019
Jiaming Luo, Yuan Cao, Regina Barzilay

* Accepted by ACL 2019 

  Access Model/Code and Paper
Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks

Jun 06, 2019
Yuan Cao, Quanquan Gu

* 23 pages 

  Access Model/Code and Paper
Gmail Smart Compose: Real-Time Assisted Writing

May 17, 2019
Mia Xu Chen, Benjamin N Lee, Gagan Bansal, Yuan Cao, Shuyuan Zhang, Justin Lu, Jackie Tsay, Yinan Wang, Andrew M. Dai, Zhifeng Chen, Timothy Sohn, Yonghui Wu


  Access Model/Code and Paper
Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networks

Apr 02, 2019
Yuan Cao, Quanquan Gu

* 54 pages. This version changes the symmetrized Gaussian initialization to standard Gaussian initialization 

  Access Model/Code and Paper
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling

Feb 21, 2019
Jonathan Shen, Patrick Nguyen, Yonghui Wu, Zhifeng Chen, Mia X. Chen, Ye Jia, Anjuli Kannan, Tara Sainath, Yuan Cao, Chung-Cheng Chiu, Yanzhang He, Jan Chorowski, Smit Hinsu, Stella Laurenzo, James Qin, Orhan Firat, Wolfgang Macherey, Suyog Gupta, Ankur Bapna, Shuyuan Zhang, Ruoming Pang, Ron J. Weiss, Rohit Prabhavalkar, Qiao Liang, Benoit Jacob, Bowen Liang, HyoukJoong Lee, Ciprian Chelba, Sébastien Jean, Bo Li, Melvin Johnson, Rohan Anil, Rajat Tibrewal, Xiaobing Liu, Akiko Eriguchi, Navdeep Jaitly, Naveen Ari, Colin Cherry, Parisa Haghani, Otavio Good, Youlong Cheng, Raziel Alvarez, Isaac Caswell, Wei-Ning Hsu, Zongheng Yang, Kuan-Chieh Wang, Ekaterina Gonina, Katrin Tomanek, Ben Vanik, Zelin Wu, Llion Jones, Mike Schuster, Yanping Huang, Dehao Chen, Kazuki Irie, George Foster, John Richardson, Klaus Macherey, Antoine Bruguier, Heiga Zen, Colin Raffel, Shankar Kumar, Kanishka Rao, David Rybach, Matthew Murray, Vijayaditya Peddinti, Maxim Krikun, Michiel A. U. Bacchiani, Thomas B. Jablin, Rob Suderman, Ian Williams, Benjamin Lee, Deepti Bhatia, Justin Carlson, Semih Yavuz, Yu Zhang, Ian McGraw, Max Galkin, Qi Ge, Golan Pundak, Chad Whipkey, Todd Wang, Uri Alon, Dmitry Lepikhin, Ye Tian, Sara Sabour, William Chan, Shubham Toshniwal, Baohua Liao, Michael Nirschl, Pat Rondon


  Access Model/Code and Paper
A Generalization Theory of Gradient Descent for Learning Over-parameterized Deep ReLU Networks

Feb 15, 2019
Yuan Cao, Quanquan Gu

* 54 pages. This version improves the sample complexity result so that it almost does not depend on the number of nodes per layer (only has a logarithmic dependence) 

  Access Model/Code and Paper
Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks

Nov 21, 2018
Difan Zou, Yuan Cao, Dongruo Zhou, Quanquan Gu

* 47 pages 

  Access Model/Code and Paper
Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translation

Nov 05, 2018
Ye Jia, Melvin Johnson, Wolfgang Macherey, Ron J. Weiss, Yuan Cao, Chung-Cheng Chiu, Naveen Ari, Stella Laurenzo, Yonghui Wu


  Access Model/Code and Paper
Hierarchical Generative Modeling for Controllable Speech Synthesis

Oct 16, 2018
Wei-Ning Hsu, Yu Zhang, Ron J. Weiss, Heiga Zen, Yonghui Wu, Yuxuan Wang, Yuan Cao, Ye Jia, Zhifeng Chen, Jonathan Shen, Patrick Nguyen, Ruoming Pang


  Access Model/Code and Paper
High Temperature Structure Detection in Ferromagnets

Sep 21, 2018
Yuan Cao, Matey Neykov, Han Liu

* 53 pages, 4 figures 

  Access Model/Code and Paper
Training Deeper Neural Machine Translation Models with Transparent Attention

Sep 04, 2018
Ankur Bapna, Mia Xu Chen, Orhan Firat, Yuan Cao, Yonghui Wu

* To appear in EMNLP 2018 

  Access Model/Code and Paper
On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization

Aug 16, 2018
Dongruo Zhou, Yiqi Tang, Ziyan Yang, Yuan Cao, Quanquan Gu

* 21 pages 

  Access Model/Code and Paper
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Oct 08, 2016
Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, Jeffrey Dean


  Access Model/Code and Paper
Training Conditional Random Fields with Natural Gradient Descent

Aug 10, 2015
Yuan Cao


  Access Model/Code and Paper
Local and Global Inference for High Dimensional Nonparanormal Graphical Models

Jun 30, 2015
Quanquan Gu, Yuan Cao, Yang Ning, Han Liu

* 58 pages, 4 figures, 2 tables 

  Access Model/Code and Paper