Alert button
Picture for Anshumali Shrivastava

Anshumali Shrivastava

Alert button

Wisdom of Committee: Distilling from Foundation Model to SpecializedApplication Model

Feb 21, 2024
Zichang Liu, Qingyun Liu, Yuening Li, Liang Liu, Anshumali Shrivastava, Shuchao Bi, Lichan Hong, Ed H. Chi, Zhe Zhao

Viaarxiv icon

Learning Scalable Structural Representations for Link Prediction with Bloom Signatures

Dec 28, 2023
Tianyi Zhang, Haoteng Yin, Rongzhe Wei, Pan Li, Anshumali Shrivastava

Viaarxiv icon

Contractive error feedback for gradient compression

Dec 13, 2023
Bingcong Li, Shuai Zheng, Parameswaran Raman, Anshumali Shrivastava, Georgios B. Giannakis

Viaarxiv icon

Adaptive Sampling for Deep Learning via Efficient Nonparametric Proxies

Nov 22, 2023
Shabnam Daghaghi, Benjamin Coleman, Benito Geordie, Anshumali Shrivastava

Viaarxiv icon

Heterogeneous federated collaborative filtering using FAIR: Federated Averaging in Random Subspaces

Nov 03, 2023
Aditya Desai, Benjamin Meisburger, Zichang Liu, Anshumali Shrivastava

Viaarxiv icon

Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time

Oct 26, 2023
Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher Re, Beidi Chen

Figure 1 for Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time
Figure 2 for Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time
Figure 3 for Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time
Figure 4 for Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time
Viaarxiv icon

In defense of parameter sharing for model-compression

Oct 17, 2023
Aditya Desai, Anshumali Shrivastava

Viaarxiv icon

Zen: Near-Optimal Sparse Tensor Synchronization for Distributed DNN Training

Sep 23, 2023
Zhuang Wang, Zhaozhuo Xu, Anshumali Shrivastava, T. S. Eugene Ng

Figure 1 for Zen: Near-Optimal Sparse Tensor Synchronization for Distributed DNN Training
Figure 2 for Zen: Near-Optimal Sparse Tensor Synchronization for Distributed DNN Training
Figure 3 for Zen: Near-Optimal Sparse Tensor Synchronization for Distributed DNN Training
Figure 4 for Zen: Near-Optimal Sparse Tensor Synchronization for Distributed DNN Training
Viaarxiv icon

CAPS: A Practical Partition Index for Filtered Similarity Search

Aug 29, 2023
Gaurav Gupta, Jonah Yi, Benjamin Coleman, Chen Luo, Vihan Lakshman, Anshumali Shrivastava

Figure 1 for CAPS: A Practical Partition Index for Filtered Similarity Search
Figure 2 for CAPS: A Practical Partition Index for Filtered Similarity Search
Figure 3 for CAPS: A Practical Partition Index for Filtered Similarity Search
Figure 4 for CAPS: A Practical Partition Index for Filtered Similarity Search
Viaarxiv icon

Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time

May 26, 2023
Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, Anshumali Shrivastava

Figure 1 for Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time
Figure 2 for Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time
Figure 3 for Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time
Figure 4 for Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time
Viaarxiv icon