Alert button
Picture for Scott Pesme

Scott Pesme

Alert button

Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks

Mar 08, 2024
Hristo Papazov, Scott Pesme, Nicolas Flammarion

Figure 1 for Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks
Figure 2 for Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks
Figure 3 for Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks
Figure 4 for Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks
Viaarxiv icon

Saddle-to-Saddle Dynamics in Diagonal Linear Networks

Apr 02, 2023
Scott Pesme, Nicolas Flammarion

Figure 1 for Saddle-to-Saddle Dynamics in Diagonal Linear Networks
Figure 2 for Saddle-to-Saddle Dynamics in Diagonal Linear Networks
Viaarxiv icon

(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability

Feb 17, 2023
Mathieu Even, Scott Pesme, Suriya Gunasekar, Nicolas Flammarion

Figure 1 for (S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability
Figure 2 for (S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability
Figure 3 for (S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability
Figure 4 for (S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability
Viaarxiv icon

Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity

Jun 17, 2021
Scott Pesme, Loucas Pillaud-Vivien, Nicolas Flammarion

Figure 1 for Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity
Figure 2 for Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity
Figure 3 for Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity
Figure 4 for Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity
Viaarxiv icon

On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent

Jul 01, 2020
Scott Pesme, Aymeric Dieuleveut, Nicolas Flammarion

Figure 1 for On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent
Figure 2 for On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent
Figure 3 for On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent
Figure 4 for On Convergence-Diagnostic based Step Sizes for Stochastic Gradient Descent
Viaarxiv icon

Online Robust Regression via SGD on the l1 loss

Jul 01, 2020
Scott Pesme, Nicolas Flammarion

Figure 1 for Online Robust Regression via SGD on the l1 loss
Viaarxiv icon