Picture for Sanjeev Arora

Sanjeev Arora

Why (and When) does Local SGD Generalize Better than SGD?

Mar 09, 2023
Xinran Gu, Kaifeng Lyu, Longbo Huang, Sanjeev Arora

Figure 1 for Why (and When) does Local SGD Generalize Better than SGD?

Figure 2 for Why (and When) does Local SGD Generalize Better than SGD?

Figure 3 for Why (and When) does Local SGD Generalize Better than SGD?

Figure 4 for Why (and When) does Local SGD Generalize Better than SGD?

* Published as a conference paper at ICLR 2023

Access Paper or Ask Questions

New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Sound

Nov 05, 2022
Arushi Gupta, Nikunj Saunshi, Dingli Yu, Kaifeng Lyu, Sanjeev Arora

Figure 1 for New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Sound

Figure 2 for New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Sound

Figure 3 for New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Sound

Figure 4 for New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Sound

* NeurIPS 2022 (Oral)

Access Paper or Ask Questions

A Kernel-Based View of Language Model Fine-Tuning

Oct 11, 2022
Sadhika Malladi, Alexander Wettig, Dingli Yu, Danqi Chen, Sanjeev Arora

Figure 1 for A Kernel-Based View of Language Model Fine-Tuning

Figure 2 for A Kernel-Based View of Language Model Fine-Tuning

Figure 3 for A Kernel-Based View of Language Model Fine-Tuning

Figure 4 for A Kernel-Based View of Language Model Fine-Tuning

* Code and pre-computed kernels are publicly available at https://github.com/princeton-nlp/LM-Kernel-FT

Access Paper or Ask Questions

Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent

Jul 08, 2022
Zhiyuan Li, Tianhao Wang, JasonD. Lee, Sanjeev Arora

Figure 1 for Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent

* 37 pages

Access Paper or Ask Questions

Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction

Jun 14, 2022
Kaifeng Lyu, Zhiyuan Li, Sanjeev Arora

Figure 1 for Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction

Figure 2 for Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction

Figure 3 for Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction

Figure 4 for Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction

* 68 pages, many figures

Access Paper or Ask Questions