Alert button

Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be

Apr 27, 2023
Frederik Kunstner, Jacques Chen, Jonathan Wilder Lavington, Mark Schmidt

Figure 1 for Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be
Figure 2 for Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be
Figure 3 for Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be
Figure 4 for Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be

Share this with someone who'll enjoy it:

View paper onarxiv icon

Share this with someone who'll enjoy it: