Alert button
Picture for Jingwen Leng

Jingwen Leng

Alert button

Fovea Transformer: Efficient Long-Context Modeling with Structured Fine-to-Coarse Attention

Nov 13, 2023
Ziwei He, Jian Yuan, Le Zhou, Jingwen Leng, Bo Jiang

Viaarxiv icon

Accelerating Generic Graph Neural Networks via Architecture, Compiler, Partition Method Co-Design

Aug 16, 2023
Shuwen Lu, Zhihui Zhang, Cong Guo, Jingwen Leng, Yangjie Zhou, Minyi Guo

Figure 1 for Accelerating Generic Graph Neural Networks via Architecture, Compiler, Partition Method Co-Design
Figure 2 for Accelerating Generic Graph Neural Networks via Architecture, Compiler, Partition Method Co-Design
Figure 3 for Accelerating Generic Graph Neural Networks via Architecture, Compiler, Partition Method Co-Design
Figure 4 for Accelerating Generic Graph Neural Networks via Architecture, Compiler, Partition Method Co-Design
Viaarxiv icon

AdaptGear: Accelerating GNN Training via Adaptive Subgraph-Level Kernels on GPUs

May 27, 2023
Yangjie Zhou, Yaoxu Song, Jingwen Leng, Zihan Liu, Weihao Cui, Zhendong Zhang, Cong Guo, Quan Chen, Li Li, Minyi Guo

Figure 1 for AdaptGear: Accelerating GNN Training via Adaptive Subgraph-Level Kernels on GPUs
Figure 2 for AdaptGear: Accelerating GNN Training via Adaptive Subgraph-Level Kernels on GPUs
Figure 3 for AdaptGear: Accelerating GNN Training via Adaptive Subgraph-Level Kernels on GPUs
Figure 4 for AdaptGear: Accelerating GNN Training via Adaptive Subgraph-Level Kernels on GPUs
Viaarxiv icon

Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator

May 24, 2023
Ziwei He, Meng Yang, Minwei Feng, Jingcheng Yin, Xinbing Wang, Jingwen Leng, Zhouhan Lin

Figure 1 for Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator
Figure 2 for Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator
Figure 3 for Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator
Figure 4 for Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator
Viaarxiv icon

Nesting Forward Automatic Differentiation for Memory-Efficient Deep Neural Network Training

Sep 22, 2022
Cong Guo, Yuxian Qiu, Jingwen Leng, Chen Zhang, Ying Cao, Quanlu Zhang, Yunxin Liu, Fan Yang, Minyi Guo

Figure 1 for Nesting Forward Automatic Differentiation for Memory-Efficient Deep Neural Network Training
Figure 2 for Nesting Forward Automatic Differentiation for Memory-Efficient Deep Neural Network Training
Figure 3 for Nesting Forward Automatic Differentiation for Memory-Efficient Deep Neural Network Training
Figure 4 for Nesting Forward Automatic Differentiation for Memory-Efficient Deep Neural Network Training
Viaarxiv icon

ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization

Aug 30, 2022
Cong Guo, Chen Zhang, Jingwen Leng, Zihan Liu, Fan Yang, Yunxin Liu, Minyi Guo, Yuhao Zhu

Figure 1 for ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization
Figure 2 for ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization
Figure 3 for ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization
Figure 4 for ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization
Viaarxiv icon

Efficient Activation Quantization via Adaptive Rounding Border for Post-Training Quantization

Aug 25, 2022
Zhengyi Li, Cong Guo, Zhanda Zhu, Yangjie Zhou, Yuxian Qiu, Xiaotian Gao, Jingwen Leng, Minyi Guo

Figure 1 for Efficient Activation Quantization via Adaptive Rounding Border for Post-Training Quantization
Figure 2 for Efficient Activation Quantization via Adaptive Rounding Border for Post-Training Quantization
Figure 3 for Efficient Activation Quantization via Adaptive Rounding Border for Post-Training Quantization
Figure 4 for Efficient Activation Quantization via Adaptive Rounding Border for Post-Training Quantization
Viaarxiv icon

SALO: An Efficient Spatial Accelerator Enabling Hybrid Sparse Attention Mechanisms for Long Sequences

Jun 29, 2022
Guan Shen, Jieru Zhao, Quan Chen, Jingwen Leng, Chao Li, Minyi Guo

Figure 1 for SALO: An Efficient Spatial Accelerator Enabling Hybrid Sparse Attention Mechanisms for Long Sequences
Figure 2 for SALO: An Efficient Spatial Accelerator Enabling Hybrid Sparse Attention Mechanisms for Long Sequences
Figure 3 for SALO: An Efficient Spatial Accelerator Enabling Hybrid Sparse Attention Mechanisms for Long Sequences
Figure 4 for SALO: An Efficient Spatial Accelerator Enabling Hybrid Sparse Attention Mechanisms for Long Sequences
Viaarxiv icon

Transkimmer: Transformer Learns to Layer-wise Skim

May 15, 2022
Yue Guan, Zhengyi Li, Jingwen Leng, Zhouhan Lin, Minyi Guo

Figure 1 for Transkimmer: Transformer Learns to Layer-wise Skim
Figure 2 for Transkimmer: Transformer Learns to Layer-wise Skim
Figure 3 for Transkimmer: Transformer Learns to Layer-wise Skim
Figure 4 for Transkimmer: Transformer Learns to Layer-wise Skim
Viaarxiv icon

SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation

Feb 14, 2022
Cong Guo, Yuxian Qiu, Jingwen Leng, Xiaotian Gao, Chen Zhang, Yunxin Liu, Fan Yang, Yuhao Zhu, Minyi Guo

Figure 1 for SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation
Figure 2 for SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation
Figure 3 for SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation
Figure 4 for SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation
Viaarxiv icon