Alert button
Picture for Ali Jadbabaie

Ali Jadbabaie

Alert button

Linear attention is (maybe) all you need (to understand transformer optimization)

Oct 02, 2023
Kwangjun Ahn, Xiang Cheng, Minhak Song, Chulhee Yun, Ali Jadbabaie, Suvrit Sra

Figure 1 for Linear attention is (maybe) all you need (to understand transformer optimization)
Figure 2 for Linear attention is (maybe) all you need (to understand transformer optimization)
Figure 3 for Linear attention is (maybe) all you need (to understand transformer optimization)
Figure 4 for Linear attention is (maybe) all you need (to understand transformer optimization)
Viaarxiv icon

Smooth Model Predictive Control with Applications to Statistical Learning

Jun 02, 2023
Kwangjun Ahn, Daniel Pfrommer, Jack Umenberger, Tobia Marcucci, Zak Mhammedi, Ali Jadbabaie

Figure 1 for Smooth Model Predictive Control with Applications to Statistical Learning
Viaarxiv icon

Convex and Non-Convex Optimization under Generalized Smoothness

Jun 02, 2023
Haochuan Li, Jian Qian, Yi Tian, Alexander Rakhlin, Ali Jadbabaie

Figure 1 for Convex and Non-Convex Optimization under Generalized Smoothness
Figure 2 for Convex and Non-Convex Optimization under Generalized Smoothness
Figure 3 for Convex and Non-Convex Optimization under Generalized Smoothness
Viaarxiv icon

Demystifying Oversmoothing in Attention-Based Graph Neural Networks

May 25, 2023
Xinyi Wu, Amir Ajorlou, Zihui Wu, Ali Jadbabaie

Figure 1 for Demystifying Oversmoothing in Attention-Based Graph Neural Networks
Viaarxiv icon

How to escape sharp minima

May 25, 2023
Kwangjun Ahn, Ali Jadbabaie, Suvrit Sra

Figure 1 for How to escape sharp minima
Figure 2 for How to escape sharp minima
Viaarxiv icon

Convergence of Adam Under Relaxed Assumptions

Apr 27, 2023
Haochuan Li, Ali Jadbabaie, Alexander Rakhlin

Viaarxiv icon

Variance-reduced Clipping for Non-convex Optimization

Mar 02, 2023
Amirhossein Reisizadeh, Haochuan Li, Subhro Das, Ali Jadbabaie

Figure 1 for Variance-reduced Clipping for Non-convex Optimization
Figure 2 for Variance-reduced Clipping for Non-convex Optimization
Figure 3 for Variance-reduced Clipping for Non-convex Optimization
Viaarxiv icon

A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networks

Dec 21, 2022
Xinyi Wu, Zhengdao Chen, William Wang, Ali Jadbabaie

Figure 1 for A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networks
Figure 2 for A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networks
Figure 3 for A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networks
Figure 4 for A Non-Asymptotic Analysis of Oversmoothing in Graph Neural Networks
Viaarxiv icon

Model Predictive Control via On-Policy Imitation Learning

Oct 17, 2022
Kwangjun Ahn, Zakaria Mhammedi, Horia Mania, Zhang-Wei Hong, Ali Jadbabaie

Figure 1 for Model Predictive Control via On-Policy Imitation Learning
Figure 2 for Model Predictive Control via On-Policy Imitation Learning
Figure 3 for Model Predictive Control via On-Policy Imitation Learning
Viaarxiv icon

On Convergence of Gradient Descent Ascent: A Tight Local Analysis

Jul 03, 2022
Haochuan Li, Farzan Farnia, Subhro Das, Ali Jadbabaie

Figure 1 for On Convergence of Gradient Descent Ascent: A Tight Local Analysis
Figure 2 for On Convergence of Gradient Descent Ascent: A Tight Local Analysis
Figure 3 for On Convergence of Gradient Descent Ascent: A Tight Local Analysis
Figure 4 for On Convergence of Gradient Descent Ascent: A Tight Local Analysis
Viaarxiv icon