Picture for Advait Gadhikar

Advait Gadhikar

Pay Attention to Small Weights

Add code
Jun 26, 2025
Viaarxiv icon

Sign-In to the Lottery: Reparameterizing Sparse Training From Scratch

Add code
Apr 17, 2025
Viaarxiv icon

Attention Is All You Need For Mixture-of-Depths Routing

Add code
Dec 30, 2024
Figure 1 for Attention Is All You Need For Mixture-of-Depths Routing
Figure 2 for Attention Is All You Need For Mixture-of-Depths Routing
Figure 3 for Attention Is All You Need For Mixture-of-Depths Routing
Figure 4 for Attention Is All You Need For Mixture-of-Depths Routing
Viaarxiv icon

Cyclic Sparse Training: Is it Enough?

Add code
Jun 07, 2024
Figure 1 for Cyclic Sparse Training: Is it Enough?
Figure 2 for Cyclic Sparse Training: Is it Enough?
Figure 3 for Cyclic Sparse Training: Is it Enough?
Figure 4 for Cyclic Sparse Training: Is it Enough?
Viaarxiv icon

Masks, Signs, And Learning Rate Rewinding

Add code
Feb 29, 2024
Viaarxiv icon

How Erdös and Rényi Win the Lottery

Add code
Oct 05, 2022
Figure 1 for How Erdös and Rényi Win the Lottery
Figure 2 for How Erdös and Rényi Win the Lottery
Figure 3 for How Erdös and Rényi Win the Lottery
Figure 4 for How Erdös and Rényi Win the Lottery
Viaarxiv icon

Dynamical Isometry for Residual Networks

Add code
Oct 05, 2022
Figure 1 for Dynamical Isometry for Residual Networks
Figure 2 for Dynamical Isometry for Residual Networks
Figure 3 for Dynamical Isometry for Residual Networks
Figure 4 for Dynamical Isometry for Residual Networks
Viaarxiv icon

Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimation

Add code
Oct 14, 2021
Figure 1 for Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimation
Figure 2 for Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimation
Figure 3 for Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimation
Figure 4 for Leveraging Spatial and Temporal Correlations in Sparsified Mean Estimation
Viaarxiv icon

A Field Guide to Federated Optimization

Add code
Jul 14, 2021
Figure 1 for A Field Guide to Federated Optimization
Figure 2 for A Field Guide to Federated Optimization
Figure 3 for A Field Guide to Federated Optimization
Figure 4 for A Field Guide to Federated Optimization
Viaarxiv icon

Adaptive Quantization of Model Updates for Communication-Efficient Federated Learning

Add code
Feb 08, 2021
Figure 1 for Adaptive Quantization of Model Updates for Communication-Efficient Federated Learning
Figure 2 for Adaptive Quantization of Model Updates for Communication-Efficient Federated Learning
Figure 3 for Adaptive Quantization of Model Updates for Communication-Efficient Federated Learning
Figure 4 for Adaptive Quantization of Model Updates for Communication-Efficient Federated Learning
Viaarxiv icon