Picture for Ivan Ilin

Ivan Ilin

PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression

Add code
May 23, 2024
Viaarxiv icon

Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity

Feb 07, 2024
Figure 1 for Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
Figure 2 for Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
Figure 3 for Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
Figure 4 for Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
Viaarxiv icon

Kimad: Adaptive Gradient Compression with Bandwidth Awareness

Dec 13, 2023
Figure 1 for Kimad: Adaptive Gradient Compression with Bandwidth Awareness
Figure 2 for Kimad: Adaptive Gradient Compression with Bandwidth Awareness
Figure 3 for Kimad: Adaptive Gradient Compression with Bandwidth Awareness
Figure 4 for Kimad: Adaptive Gradient Compression with Bandwidth Awareness
Viaarxiv icon