Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for H. Brendan McMahan

A Field Guide to Federated Optimization


Jul 14, 2021
Jianyu Wang, Zachary Charles, Zheng Xu, Gauri Joshi, H. Brendan McMahan, Blaise Aguera y Arcas, Maruan Al-Shedivat, Galen Andrew, Salman Avestimehr, Katharine Daly, Deepesh Data, Suhas Diggavi, Hubert Eichner, Advait Gadhikar, Zachary Garrett, Antonious M. Girgis, Filip Hanzely, Andrew Hard, Chaoyang He, Samuel Horvath, Zhouyuan Huo, Alex Ingerman, Martin Jaggi, Tara Javidi, Peter Kairouz, Satyen Kale, Sai Praneeth Karimireddy, Jakub Konecny, Sanmi Koyejo, Tian Li, Luyang Liu, Mehryar Mohri, Hang Qi, Sashank J. Reddi, Peter Richtarik, Karan Singhal, Virginia Smith, Mahdi Soltanolkotabi, Weikang Song, Ananda Theertha Suresh, Sebastian U. Stich, Ameet Talwalkar, Hongyi Wang, Blake Woodworth, Shanshan Wu, Felix X. Yu, Honglin Yuan, Manzil Zaheer, Mi Zhang, Tong Zhang, Chunxiang Zheng, Chen Zhu, Wennan Zhu


  Access Paper or Ask Questions

Training Production Language Models without Memorizing User Data


Sep 21, 2020
Swaroop Ramaswamy, Om Thakkar, Rajiv Mathews, Galen Andrew, H. Brendan McMahan, Fran├žoise Beaufays


  Access Paper or Ask Questions

Privacy Amplification via Random Check-Ins


Jul 30, 2020
Borja Balle, Peter Kairouz, H. Brendan McMahan, Om Thakkar, Abhradeep Thakurta

* Updated proof for $(\epsilon_0, \delta_0)$-DP local randomizers 

  Access Paper or Ask Questions

Adaptive Federated Optimization


Feb 29, 2020
Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Kone─Źn├Ż, Sanjiv Kumar, H. Brendan McMahan


  Access Paper or Ask Questions

Is Local SGD Better than Minibatch SGD?


Feb 18, 2020
Blake Woodworth, Kumar Kshitij Patel, Sebastian U. Stich, Zhen Dai, Brian Bullins, H. Brendan McMahan, Ohad Shamir, Nathan Srebro

* 29 pages 

  Access Paper or Ask Questions

Advances and Open Problems in Federated Learning


Dec 10, 2019
Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aur├ęlien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D'Oliveira, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adri├á Gasc├│n, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Kone─Źn├Ż, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancr├Ęde Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer ├ľzg├╝r, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tram├Ęr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, Sen Zhao


  Access Paper or Ask Questions

Can You Really Backdoor Federated Learning?


Dec 02, 2019
Ziteng Sun, Peter Kairouz, Ananda Theertha Suresh, H. Brendan McMahan

* To appear at the 2nd International Workshop on Federated Learning for Data Privacy and Confidentiality at NeurIPS 2019 

  Access Paper or Ask Questions

Generative Models for Effective ML on Private, Decentralized Datasets


Nov 15, 2019
Sean Augenstein, H. Brendan McMahan, Daniel Ramage, Swaroop Ramaswamy, Peter Kairouz, Mingqing Chen, Rajiv Mathews, Blaise Aguera y Arcas

* 27 pages, 8 figures 

  Access Paper or Ask Questions

Differentially Private Learning with Adaptive Clipping


May 09, 2019
Om Thakkar, Galen Andrew, H. Brendan McMahan


  Access Paper or Ask Questions

SysML: The New Frontier of Machine Learning Systems


May 01, 2019
Alexander Ratner, Dan Alistarh, Gustavo Alonso, David G. Andersen, Peter Bailis, Sarah Bird, Nicholas Carlini, Bryan Catanzaro, Jennifer Chayes, Eric Chung, Bill Dally, Jeff Dean, Inderjit S. Dhillon, Alexandros Dimakis, Pradeep Dubey, Charles Elkan, Grigori Fursin, Gregory R. Ganger, Lise Getoor, Phillip B. Gibbons, Garth A. Gibson, Joseph E. Gonzalez, Justin Gottschlich, Song Han, Kim Hazelwood, Furong Huang, Martin Jaggi, Kevin Jamieson, Michael I. Jordan, Gauri Joshi, Rania Khalaf, Jason Knight, Jakub Kone─Źn├Ż, Tim Kraska, Arun Kumar, Anastasios Kyrillidis, Aparna Lakshmiratan, Jing Li, Samuel Madden, H. Brendan McMahan, Erik Meijer, Ioannis Mitliagkas, Rajat Monga, Derek Murray, Kunle Olukotun, Dimitris Papailiopoulos, Gennady Pekhimenko, Theodoros Rekatsinas, Afshin Rostamizadeh, Christopher R├ę, Christopher De Sa, Hanie Sedghi, Siddhartha Sen, Virginia Smith, Alex Smola, Dawn Song, Evan Sparks, Ion Stoica, Vivienne Sze, Madeleine Udell, Joaquin Vanschoren, Shivaram Venkataraman, Rashmi Vinayak, Markus Weimer, Andrew Gordon Wilson, Eric Xing, Matei Zaharia, Ce Zhang, Ameet Talwalkar


  Access Paper or Ask Questions

Semi-Cyclic Stochastic Gradient Descent


Apr 23, 2019
Hubert Eichner, Tomer Koren, H. Brendan McMahan, Nathan Srebro, Kunal Talwar


  Access Paper or Ask Questions

Towards Federated Learning at Scale: System Design


Mar 22, 2019
Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Kone─Źn├Ż, Stefano Mazzocchi, H. Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, Jason Roselander


  Access Paper or Ask Questions

LEAF: A Benchmark for Federated Settings


Jan 09, 2019
Sebastian Caldas, Peter Wu, Tian Li, Jakub Kone─Źn├Ż, H. Brendan McMahan, Virginia Smith, Ameet Talwalkar


  Access Paper or Ask Questions

Expanding the Reach of Federated Learning by Reducing Client Resource Requirements


Jan 08, 2019
Sebastian Caldas, Jakub Kone─Źny, H. Brendan McMahan, Ameet Talwalkar


  Access Paper or Ask Questions

A General Approach to Adding Differential Privacy to Iterative Training Procedures


Dec 15, 2018
H. Brendan McMahan, Galen Andrew

* Presented at NeurIPS 2018 workshop on Privacy Preserving Machine Learning 

  Access Paper or Ask Questions

Learning Differentially Private Recurrent Language Models


Feb 24, 2018
H. Brendan McMahan, Daniel Ramage, Kunal Talwar, Li Zhang

* Camera-ready ICLR 2018 version, minor edits from previous 

  Access Paper or Ask Questions

Federated Learning: Strategies for Improving Communication Efficiency


Oct 30, 2017
Jakub Kone─Źn├Ż, H. Brendan McMahan, Felix X. Yu, Peter Richt├írik, Ananda Theertha Suresh, Dave Bacon


  Access Paper or Ask Questions

Distributed Mean Estimation with Limited Communication


Sep 25, 2017
Ananda Theertha Suresh, Felix X. Yu, Sanjiv Kumar, H. Brendan McMahan


  Access Paper or Ask Questions

On the Protection of Private Information in Machine Learning Systems: Two Recent Approaches


Aug 26, 2017
Mart├şn Abadi, ├Ülfar Erlingsson, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Nicolas Papernot, Kunal Talwar, Li Zhang

* IEEE 30th Computer Security Foundations Symposium (CSF), pages 1--6, 2017 

  Access Paper or Ask Questions

Communication-Efficient Learning of Deep Networks from Decentralized Data


Feb 28, 2017
H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, Blaise Ag├╝era y Arcas

* Proceedings of the 20 th International Conference on Artificial Intelligence and Statistics (AISTATS) 2017. JMLR: W&CP volume 54 
* This version updates the large-scale LSTM experiments, along with other minor changes. In earlier versions, an inconsistency in our implementation of FedSGD caused us to report much lower learning rates for the large-scale LSTM. We reran these experiments, and also found that fewer local epochs offers better performance, leading to slightly better results for FedAvg than previously reported 

  Access Paper or Ask Questions

Practical Secure Aggregation for Federated Learning on User-Held Data


Nov 14, 2016
Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, Karn Seth

* 5 pages, 1 figure. To appear at the NIPS 2016 workshop on Private Multi-Party Machine Learning 

  Access Paper or Ask Questions

Deep Learning with Differential Privacy


Oct 24, 2016
Mart├şn Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, Li Zhang


  Access Paper or Ask Questions

Federated Optimization: Distributed Machine Learning for On-Device Intelligence


Oct 08, 2016
Jakub Kone─Źn├Ż, H. Brendan McMahan, Daniel Ramage, Peter Richt├írik

* 38 pages 

  Access Paper or Ask Questions

A Survey of Algorithms and Analysis for Adaptive Online Learning


Nov 09, 2015
H. Brendan McMahan


  Access Paper or Ask Questions

Unconstrained Online Linear Learning in Hilbert Spaces: Minimax Algorithms and Normal Approximations


May 21, 2014
H. Brendan McMahan, Francesco Orabona

* Proceedings of the 27th Annual Conference on Learning Theory (COLT 2014) 

  Access Paper or Ask Questions