Song Han

University of Connecticut

HAT: Hardware-Aware Transformers for Efficient Natural Language Processing

May 28, 2020
Hanrui Wang, Zhanghao Wu, Zhijian Liu, Han Cai, Ligeng Zhu, Chuang Gan, Song Han

* Accepted to ACL 2020. 14 pages, 12 figures. Code available at http://github.com/mit-han-lab/hardware-aware-transformers.git 
  Access Model/Code and Paper
MicroNet for Efficient Language Modeling

May 16, 2020
Zhongxia Yan, Hanrui Wang, Demi Guo, Song Han

* Accepted by PMLR 
  Access Model/Code and Paper
GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learning

Apr 30, 2020
Hanrui Wang, Kuan Wang, Jiacheng Yang, Linxiao Shen, Nan Sun, Hae-Seung Lee, Song Han

* Accepted to the 57th Design Automation Conference (DAC 2020); 6 pages, 8 figures 
  Access Model/Code and Paper
Lite Transformer with Long-Short Range Attention

Apr 24, 2020
Zhanghao Wu, Zhijian Liu, Ji Lin, Yujun Lin, Song Han

* ICLR 2020. The first two authors contributed equally to this work 
  Access Model/Code and Paper
GAN Compression: Efficient Architectures for Interactive Conditional GANs

Mar 19, 2020
Muyang Li, Ji Lin, Yaoyao Ding, Zhijian Liu, Jun-Yan Zhu, Song Han


  Access Model/Code and Paper
Training Kinetics in 15 Minutes: Large-scale Distributed Training on Videos

Oct 01, 2019
Ji Lin, Chuang Gan, Song Han


  Access Model/Code and Paper
Once for All: Train One Network and Specialize it for Efficient Deployment

Aug 26, 2019
Han Cai, Chuang Gan, Song Han


  Access Model/Code and Paper
Point-Voxel CNN for Efficient 3D Deep Learning

Jul 08, 2019
Zhijian Liu, Haotian Tang, Yujun Lin, Song Han

* The first two authors contributed equally to this work 
  Access Model/Code and Paper
Deep Leakage from Gradients

Jun 21, 2019
Ligeng Zhu, Zhijian Liu, Song Han


  Access Model/Code and Paper
SysML: The New Frontier of Machine Learning Systems

May 01, 2019
Alexander Ratner, Dan Alistarh, Gustavo Alonso, David G. Andersen, Peter Bailis, Sarah Bird, Nicholas Carlini, Bryan Catanzaro, Jennifer Chayes, Eric Chung, Bill Dally, Jeff Dean, Inderjit S. Dhillon, Alexandros Dimakis, Pradeep Dubey, Charles Elkan, Grigori Fursin, Gregory R. Ganger, Lise Getoor, Phillip B. Gibbons, Garth A. Gibson, Joseph E. Gonzalez, Justin Gottschlich, Song Han, Kim Hazelwood, Furong Huang, Martin Jaggi, Kevin Jamieson, Michael I. Jordan, Gauri Joshi, Rania Khalaf, Jason Knight, Jakub Konečný, Tim Kraska, Arun Kumar, Anastasios Kyrillidis, Aparna Lakshmiratan, Jing Li, Samuel Madden, H. Brendan McMahan, Erik Meijer, Ioannis Mitliagkas, Rajat Monga, Derek Murray, Kunle Olukotun, Dimitris Papailiopoulos, Gennady Pekhimenko, Theodoros Rekatsinas, Afshin Rostamizadeh, Christopher Ré, Christopher De Sa, Hanie Sedghi, Siddhartha Sen, Virginia Smith, Alex Smola, Dawn Song, Evan Sparks, Ion Stoica, Vivienne Sze, Madeleine Udell, Joaquin Vanschoren, Shivaram Venkataraman, Rashmi Vinayak, Markus Weimer, Andrew Gordon Wilson, Eric Xing, Matei Zaharia, Ce Zhang, Ameet Talwalkar


  Access Model/Code and Paper
Design Automation for Efficient Deep Learning Computing

Apr 24, 2019
Song Han, Han Cai, Ligeng Zhu, Ji Lin, Kuan Wang, Zhijian Liu, Yujun Lin


  Access Model/Code and Paper
Defensive Quantization: When Efficiency Meets Robustness

Apr 17, 2019
Ji Lin, Chuang Gan, Song Han


  Access Model/Code and Paper
Learning to Design Circuits

Jan 17, 2019
Hanrui Wang, Jiacheng Yang, Hae-Seung Lee, Song Han

* The first two authors contributed equally to this work 
  Access Model/Code and Paper
HAQ: Hardware-Aware Automated Quantization

Dec 06, 2018
Kuan Wang, Zhijian Liu, Yujun Lin, Ji Lin, Song Han

* The first three authors contributed equally to this work. Project page: https://hanlab.mit.edu/projects/haq/ 
  Access Model/Code and Paper
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware

Dec 02, 2018
Han Cai, Ligeng Zhu, Song Han

* Submitted to ICLR 2019 
  Access Model/Code and Paper
Temporal Shift Module for Efficient Video Understanding

Nov 20, 2018
Ji Lin, Chuang Gan, Song Han

* Our framework ranks the first on both Something-Something V1 and V2 leaderboards upon this paper's submission. Measured on P100 GPU, our single model achieved 1.8% higher accuracy at 8x lower latency and 12x higher throughput compared to I3D 
  Access Model/Code and Paper
AMC: AutoML for Model Compression and Acceleration on Mobile Devices

Aug 26, 2018
Yihui He, Ji Lin, Zhijian Liu, Hanrui Wang, Li-Jia Li, Song Han


  Access Model/Code and Paper
Fast inference of deep neural networks in FPGAs for particle physics

Jun 28, 2018
Javier Duarte, Song Han, Philip Harris, Sergo Jindariani, Edward Kreinar, Benjamin Kreis, Jennifer Ngadiuba, Maurizio Pierini, Ryan Rivera, Nhan Tran, Zhenbin Wu

* 22 pages, 17 figures, 2 tables, JINST revision 
  Access Model/Code and Paper
Path-Level Network Transformation for Efficient Architecture Search

Jun 07, 2018
Han Cai, Jiacheng Yang, Weinan Zhang, Song Han, Yong Yu

* ICML 2018 
  Access Model/Code and Paper
Efficient Sparse-Winograd Convolutional Neural Networks

Feb 18, 2018
Xingyu Liu, Jeff Pool, Song Han, William J. Dally

* Published as a conference paper at ICLR 2018 
  Access Model/Code and Paper
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

Feb 05, 2018
Yujun Lin, Song Han, Huizi Mao, Yu Wang, William J. Dally

* ICLR 2018 
 * we find 99.9% of the gradient exchange in distributed SGD is redundant; we reduce the communication bandwidth by two orders of magnitude without losing accuracy 
  Access Model/Code and Paper
Exploring the Regularity of Sparse Structure in Convolutional Neural Networks

Jun 05, 2017
Huizi Mao, Song Han, Jeff Pool, Wenshuo Li, Xingyu Liu, Yu Wang, William J. Dally

* submitted to NIPS 2017 
  Access Model/Code and Paper
Deep Generative Adversarial Networks for Compressed Sensing Automates MRI

May 31, 2017
Morteza Mardani, Enhao Gong, Joseph Y. Cheng, Shreyas Vasanawala, Greg Zaharchuk, Marcus Alley, Neil Thakur, Song Han, William Dally, John M. Pauly, Lei Xing


  Access Model/Code and Paper
Trained Ternary Quantization

Feb 23, 2017
Chenzhuo Zhu, Song Han, Huizi Mao, William J. Dally

* Accepted for Poster Presentation on ICLR 2017 
  Access Model/Code and Paper
DSD: Dense-Sparse-Dense Training for Deep Neural Networks

Feb 21, 2017
Song Han, Jeff Pool, Sharan Narang, Huizi Mao, Enhao Gong, Shijian Tang, Erich Elsen, Peter Vajda, Manohar Paluri, John Tran, Bryan Catanzaro, William J. Dally

* Published as a conference paper at ICLR 2017 
  Access Model/Code and Paper
ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA

Feb 20, 2017
Song Han, Junlong Kang, Huizi Mao, Yiming Hu, Xin Li, Yubin Li, Dongliang Xie, Hong Luo, Song Yao, Yu Wang, Huazhong Yang, William J. Dally

* Accepted as full paper in FPGA'17, Monterey, CA; Also appeared at 1st International Workshop on Efficient Methods for Deep Neural Networks at NIPS 2016, Barcelona, Spain 
  Access Model/Code and Paper
Classification of Neurological Gait Disorders Using Multi-task Feature Learning

Feb 01, 2017
Ioannis Papavasileiou, Wenlong Zhang, Xin Wang, Jinbo Bi, Li Zhang, Song Han

* shorter version of this paper is submitted to CHASE '17 conference 
  Access Model/Code and Paper
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size

Nov 04, 2016
Forrest N. Iandola, Song Han, Matthew W. Moskewicz, Khalid Ashraf, William J. Dally, Kurt Keutzer

* In ICLR Format 
  Access Model/Code and Paper