Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Sebastian U. Stich

A Field Guide to Federated Optimization


Jul 14, 2021
Jianyu Wang, Zachary Charles, Zheng Xu, Gauri Joshi, H. Brendan McMahan, Blaise Aguera y Arcas, Maruan Al-Shedivat, Galen Andrew, Salman Avestimehr, Katharine Daly, Deepesh Data, Suhas Diggavi, Hubert Eichner, Advait Gadhikar, Zachary Garrett, Antonious M. Girgis, Filip Hanzely, Andrew Hard, Chaoyang He, Samuel Horvath, Zhouyuan Huo, Alex Ingerman, Martin Jaggi, Tara Javidi, Peter Kairouz, Satyen Kale, Sai Praneeth Karimireddy, Jakub Konecny, Sanmi Koyejo, Tian Li, Luyang Liu, Mehryar Mohri, Hang Qi, Sashank J. Reddi, Peter Richtarik, Karan Singhal, Virginia Smith, Mahdi Soltanolkotabi, Weikang Song, Ananda Theertha Suresh, Sebastian U. Stich, Ameet Talwalkar, Hongyi Wang, Blake Woodworth, Shanshan Wu, Felix X. Yu, Honglin Yuan, Manzil Zaheer, Mi Zhang, Tong Zhang, Chunxiang Zheng, Chen Zhu, Wennan Zhu


  Access Paper or Ask Questions

Simultaneous Training of Partially Masked Neural Networks


Jun 16, 2021
Amirkeivan Mohtashami, Martin Jaggi, Sebastian U. Stich


  Access Paper or Ask Questions

Critical Parameters for Scalable Distributed Learning with Large Batches and Asynchronous Updates


Mar 03, 2021
Sebastian U. Stich, Amirkeivan Mohtashami, Martin Jaggi

* Proceedings of the 24th International Conference on Artificial Intelligence and Statistics (AISTATS) 2021 

  Access Paper or Ask Questions

Consensus Control for Decentralized Deep Learning


Feb 09, 2021
Lingjing Kong, Tao Lin, Anastasia Koloskova, Martin Jaggi, Sebastian U. Stich

* LK and TL contribute equally 

  Access Paper or Ask Questions

Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data


Feb 09, 2021
Tao Lin, Sai Praneeth Karimireddy, Sebastian U. Stich, Martin Jaggi


  Access Paper or Ask Questions

A Linearly Convergent Algorithm for Decentralized Optimization: Sending Less Bits for Free!


Nov 03, 2020
Dmitry Kovalev, Anastasia Koloskova, Martin Jaggi, Peter Richtarik, Sebastian U. Stich


  Access Paper or Ask Questions

On Communication Compression for Distributed Optimization on Heterogeneous Data


Sep 04, 2020
Sebastian U. Stich


  Access Paper or Ask Questions

Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning


Aug 08, 2020
Sai Praneeth Karimireddy, Martin Jaggi, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, Ananda Theertha Suresh


  Access Paper or Ask Questions

Analysis of SGD with Biased Gradient Estimators


Jul 31, 2020
Ahmad Ajalloeian, Sebastian U. Stich

* Accepted to ICML 2020 Workshop "Beyond First Order Methods in ML Systems" 

  Access Paper or Ask Questions

Dynamic Model Pruning with Feedback


Jun 12, 2020
Tao Lin, Sebastian U. Stich, Luis Barba, Daniil Dmitriev, Martin Jaggi

* appearing at ICLR 2020 

  Access Paper or Ask Questions

Ensemble Distillation for Robust Model Fusion in Federated Learning


Jun 12, 2020
Tao Lin, Lingjing Kong, Sebastian U. Stich, Martin Jaggi


  Access Paper or Ask Questions

Extrapolation for Large-batch Training in Deep Learning


Jun 10, 2020
Tao Lin, Lingjing Kong, Sebastian U. Stich, Martin Jaggi


  Access Paper or Ask Questions

A Unified Theory of Decentralized SGD with Changing Topology and Local Updates


Mar 23, 2020
Anastasia Koloskova, Nicolas Loizou, Sadra Boreiri, Martin Jaggi, Sebastian U. Stich


  Access Paper or Ask Questions

Is Local SGD Better than Minibatch SGD?


Feb 18, 2020
Blake Woodworth, Kumar Kshitij Patel, Sebastian U. Stich, Zhen Dai, Brian Bullins, H. Brendan McMahan, Ohad Shamir, Nathan Srebro

* 29 pages 

  Access Paper or Ask Questions

Advances and Open Problems in Federated Learning


Dec 10, 2019
Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D'Oliveira, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konečný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, Sen Zhao


  Access Paper or Ask Questions

SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning


Oct 14, 2019
Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, Ananda Theertha Suresh


  Access Paper or Ask Questions

The Error-Feedback Framework: Better Rates for SGD with Delayed Gradients and Compressed Communication


Sep 11, 2019
Sebastian U. Stich, Sai Praneeth Karimireddy


  Access Paper or Ask Questions

Decentralized Deep Learning with Arbitrary Communication Compression


Jul 22, 2019
Anastasia Koloskova, Tao Lin, Sebastian U. Stich, Martin Jaggi


  Access Paper or Ask Questions

Unified Optimal Analysis of the (Stochastic) Gradient Method


Jul 09, 2019
Sebastian U. Stich

* 9 pages 

  Access Paper or Ask Questions

Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication


Feb 01, 2019
Anastasia Koloskova, Sebastian U. Stich, Martin Jaggi


  Access Paper or Ask Questions

Error Feedback Fixes SignSGD and other Gradient Compression Schemes


Jan 28, 2019
Sai Praneeth Karimireddy, Quentin Rebjock, Sebastian U. Stich, Martin Jaggi


  Access Paper or Ask Questions

Don't Use Large Mini-Batches, Use Local SGD


Oct 21, 2018
Tao Lin, Sebastian U. Stich, Martin Jaggi


  Access Paper or Ask Questions

Efficient Greedy Coordinate Descent for Composite Problems


Oct 16, 2018
Sai Praneeth Karimireddy, Anastasia Koloskova, Sebastian U. Stich, Martin Jaggi

* 44 pages, 17 figures, 3 tables 

  Access Paper or Ask Questions

k-SVRG: Variance Reduction for Large Scale Optimization


Oct 16, 2018
Anant Raj, Sebastian U. Stich

* The title of the previous version of the manuscript was "SVRG meets SAGA: k-SVRG A Tale of Limited Memory" 

  Access Paper or Ask Questions

Sparsified SGD with Memory


Sep 20, 2018
Sebastian U. Stich, Jean-Baptiste Cordonnier, Martin Jaggi

* to appear at NIPS 2018 

  Access Paper or Ask Questions

On Matching Pursuit and Coordinate Descent


Jul 02, 2018
Francesco Locatello, Anant Raj, Sai Praneeth Karimireddy, Gunnar Rätsch, Bernhard Schölkopf, Sebastian U. Stich, Martin Jaggi

* ICML 2018 - Proceedings of the 35th International Conference on Machine Learning 

  Access Paper or Ask Questions

Local SGD Converges Fast and Communicates Little


Jun 25, 2018
Sebastian U. Stich

* preprint, 12 pages 

  Access Paper or Ask Questions

Global linear convergence of Newton's method without strong-convexity or Lipschitz gradients


Jun 01, 2018
Sai Praneeth Karimireddy, Sebastian U. Stich, Martin Jaggi

* 19 pages 

  Access Paper or Ask Questions

Safe Adaptive Importance Sampling


Nov 07, 2017
Sebastian U. Stich, Anant Raj, Martin Jaggi

* To appear at NIPS 2017 

  Access Paper or Ask Questions

Approximate Steepest Coordinate Descent


Jun 26, 2017
Sebastian U. Stich, Anant Raj, Martin Jaggi

* appearing at ICML 2017 

  Access Paper or Ask Questions