Alert button
Picture for Zhenyu Zhang

Zhenyu Zhang

Alert button

Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity

Oct 08, 2023
Lu Yin, You Wu, Zhenyu Zhang, Cheng-Yu Hsieh, Yaqing Wang, Yiling Jia, Mykola Pechenizkiy, Yi Liang, Zhangyang Wang, Shiwei Liu

Figure 1 for Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity
Figure 2 for Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity
Figure 3 for Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity
Figure 4 for Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity
Viaarxiv icon

JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention

Oct 03, 2023
Yuandong Tian, Yiping Wang, Zhenyu Zhang, Beidi Chen, Simon Du

Viaarxiv icon

Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy

Oct 02, 2023
Pingzhi Li, Zhenyu Zhang, Prateek Yadav, Yi-Lin Sung, Yu Cheng, Mohit Bansal, Tianlong Chen

Figure 1 for Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy
Figure 2 for Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy
Figure 3 for Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy
Figure 4 for Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy
Viaarxiv icon

RigNet++: Efficient Repetitive Image Guided Network for Depth Completion

Sep 15, 2023
Zhiqiang Yan, Xiang Li, Zhenyu Zhang, Jun Li, Jian Yang

Figure 1 for RigNet++: Efficient Repetitive Image Guided Network for Depth Completion
Figure 2 for RigNet++: Efficient Repetitive Image Guided Network for Depth Completion
Figure 3 for RigNet++: Efficient Repetitive Image Guided Network for Depth Completion
Figure 4 for RigNet++: Efficient Repetitive Image Guided Network for Depth Completion
Viaarxiv icon

A study on the impact of pre-trained model on Just-In-Time defect prediction

Sep 05, 2023
Yuxiang Guo, Xiaopeng Gao, Zhenyu Zhang, W. K. Chan, Bo Jiang

Figure 1 for A study on the impact of pre-trained model on Just-In-Time defect prediction
Figure 2 for A study on the impact of pre-trained model on Just-In-Time defect prediction
Figure 3 for A study on the impact of pre-trained model on Just-In-Time defect prediction
Figure 4 for A study on the impact of pre-trained model on Just-In-Time defect prediction
Viaarxiv icon

AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimization

Aug 19, 2023
Kun Wang, Zhiqiang Yan, Huang Tian, Zhenyu Zhang, Xiang Li, Jun Li, Jian Yang

Figure 1 for AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimization
Figure 2 for AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimization
Figure 3 for AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimization
Figure 4 for AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimization
Viaarxiv icon

H$_2$O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models

Jul 19, 2023
Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, Beidi Chen

Figure 1 for H$_2$O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
Figure 2 for H$_2$O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
Figure 3 for H$_2$O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
Figure 4 for H$_2$O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
Viaarxiv icon

MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling

Jun 29, 2023
Zhenyu Zhang, Wenhao Chai, Zhongyu Jiang, Tian Ye, Mingli Song, Jenq-Neng Hwang, Gaoang Wang

Figure 1 for MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling
Figure 2 for MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling
Figure 3 for MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling
Figure 4 for MPM: A Unified 2D-3D Human Pose Representation via Masked Pose Modeling
Viaarxiv icon