Picture for Soufiane Hayou

Soufiane Hayou

The Impact of Initialization on LoRA Finetuning Dynamics

Add code
Jun 12, 2024
Viaarxiv icon

How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse

Add code
Apr 07, 2024
Figure 1 for How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
Figure 2 for How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
Figure 3 for How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
Figure 4 for How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse
Viaarxiv icon

LoRA+: Efficient Low Rank Adaptation of Large Models

Add code
Feb 19, 2024
Figure 1 for LoRA+: Efficient Low Rank Adaptation of Large Models
Figure 2 for LoRA+: Efficient Low Rank Adaptation of Large Models
Figure 3 for LoRA+: Efficient Low Rank Adaptation of Large Models
Figure 4 for LoRA+: Efficient Low Rank Adaptation of Large Models
Viaarxiv icon

Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks

Add code
Oct 12, 2023
Figure 1 for Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks
Figure 2 for Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks
Figure 3 for Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks
Figure 4 for Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks
Viaarxiv icon

Commutative Width and Depth Scaling in Deep Neural Networks

Add code
Oct 02, 2023
Viaarxiv icon

Leave-one-out Distinguishability in Machine Learning

Add code
Sep 29, 2023
Viaarxiv icon

On the Connection Between Riemann Hypothesis and a Special Class of Neural Networks

Add code
Sep 17, 2023
Figure 1 for On the Connection Between Riemann Hypothesis and a Special Class of Neural Networks
Figure 2 for On the Connection Between Riemann Hypothesis and a Special Class of Neural Networks
Viaarxiv icon

Data pruning and neural scaling laws: fundamental limitations of score-based algorithms

Add code
Feb 14, 2023
Figure 1 for Data pruning and neural scaling laws: fundamental limitations of score-based algorithms
Figure 2 for Data pruning and neural scaling laws: fundamental limitations of score-based algorithms
Figure 3 for Data pruning and neural scaling laws: fundamental limitations of score-based algorithms
Figure 4 for Data pruning and neural scaling laws: fundamental limitations of score-based algorithms
Viaarxiv icon

Width and Depth Limits Commute in Residual Networks

Add code
Feb 01, 2023
Figure 1 for Width and Depth Limits Commute in Residual Networks
Figure 2 for Width and Depth Limits Commute in Residual Networks
Figure 3 for Width and Depth Limits Commute in Residual Networks
Figure 4 for Width and Depth Limits Commute in Residual Networks
Viaarxiv icon

On the infinite-depth limit of finite-width neural networks

Add code
Oct 03, 2022
Figure 1 for On the infinite-depth limit of finite-width neural networks
Figure 2 for On the infinite-depth limit of finite-width neural networks
Figure 3 for On the infinite-depth limit of finite-width neural networks
Figure 4 for On the infinite-depth limit of finite-width neural networks
Viaarxiv icon