Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for H. Brendan McMahan

H. Brendan McMahan

A Field Guide to Federated Optimization


Jul 14, 2021
Jianyu Wang, Zachary Charles, Zheng Xu, Gauri Joshi, H. Brendan McMahan, Blaise Aguera y Arcas, Maruan Al-Shedivat, Galen Andrew, Salman Avestimehr, Katharine Daly, Deepesh Data, Suhas Diggavi, Hubert Eichner, Advait Gadhikar, Zachary Garrett, Antonious M. Girgis, Filip Hanzely, Andrew Hard, Chaoyang He, Samuel Horvath, Zhouyuan Huo, Alex Ingerman, Martin Jaggi, Tara Javidi, Peter Kairouz, Satyen Kale, Sai Praneeth Karimireddy, Jakub Konecny, Sanmi Koyejo, Tian Li, Luyang Liu, Mehryar Mohri, Hang Qi, Sashank J. Reddi, Peter Richtarik, Karan Singhal, Virginia Smith, Mahdi Soltanolkotabi, Weikang Song, Ananda Theertha Suresh, Sebastian U. Stich, Ameet Talwalkar, Hongyi Wang, Blake Woodworth, Shanshan Wu, Felix X. Yu, Honglin Yuan, Manzil Zaheer, Mi Zhang, Tong Zhang, Chunxiang Zheng, Chen Zhu, Wennan Zhu

Figure 1 for A Field Guide to Federated Optimization
Figure 2 for A Field Guide to Federated Optimization
Figure 3 for A Field Guide to Federated Optimization
Figure 4 for A Field Guide to Federated Optimization

  Access Paper or Ask Questions

Training Production Language Models without Memorizing User Data


Sep 21, 2020
Swaroop Ramaswamy, Om Thakkar, Rajiv Mathews, Galen Andrew, H. Brendan McMahan, Françoise Beaufays

Figure 1 for Training Production Language Models without Memorizing User Data
Figure 2 for Training Production Language Models without Memorizing User Data
Figure 3 for Training Production Language Models without Memorizing User Data
Figure 4 for Training Production Language Models without Memorizing User Data

  Access Paper or Ask Questions

Privacy Amplification via Random Check-Ins


Jul 30, 2020
Borja Balle, Peter Kairouz, H. Brendan McMahan, Om Thakkar, Abhradeep Thakurta

Figure 1 for Privacy Amplification via Random Check-Ins
Figure 2 for Privacy Amplification via Random Check-Ins
Figure 3 for Privacy Amplification via Random Check-Ins
* Updated proof for $(\epsilon_0, \delta_0)$-DP local randomizers 
  Access Paper or Ask Questions

Adaptive Federated Optimization


Feb 29, 2020
Sashank Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, H. Brendan McMahan

Figure 1 for Adaptive Federated Optimization
Figure 2 for Adaptive Federated Optimization
Figure 3 for Adaptive Federated Optimization
Figure 4 for Adaptive Federated Optimization

  Access Paper or Ask Questions

Is Local SGD Better than Minibatch SGD?


Feb 18, 2020
Blake Woodworth, Kumar Kshitij Patel, Sebastian U. Stich, Zhen Dai, Brian Bullins, H. Brendan McMahan, Ohad Shamir, Nathan Srebro

Figure 1 for Is Local SGD Better than Minibatch SGD?
Figure 2 for Is Local SGD Better than Minibatch SGD?
Figure 3 for Is Local SGD Better than Minibatch SGD?
* 29 pages 
  Access Paper or Ask Questions

Advances and Open Problems in Federated Learning


Dec 10, 2019
Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Keith Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D'Oliveira, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konečný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, Sen Zhao

Figure 1 for Advances and Open Problems in Federated Learning
Figure 2 for Advances and Open Problems in Federated Learning
Figure 3 for Advances and Open Problems in Federated Learning
Figure 4 for Advances and Open Problems in Federated Learning

  Access Paper or Ask Questions

Can You Really Backdoor Federated Learning?


Dec 02, 2019
Ziteng Sun, Peter Kairouz, Ananda Theertha Suresh, H. Brendan McMahan

Figure 1 for Can You Really Backdoor Federated Learning?
Figure 2 for Can You Really Backdoor Federated Learning?
Figure 3 for Can You Really Backdoor Federated Learning?
Figure 4 for Can You Really Backdoor Federated Learning?
* To appear at the 2nd International Workshop on Federated Learning for Data Privacy and Confidentiality at NeurIPS 2019 
  Access Paper or Ask Questions

Generative Models for Effective ML on Private, Decentralized Datasets


Nov 15, 2019
Sean Augenstein, H. Brendan McMahan, Daniel Ramage, Swaroop Ramaswamy, Peter Kairouz, Mingqing Chen, Rajiv Mathews, Blaise Aguera y Arcas

Figure 1 for Generative Models for Effective ML on Private, Decentralized Datasets
Figure 2 for Generative Models for Effective ML on Private, Decentralized Datasets
Figure 3 for Generative Models for Effective ML on Private, Decentralized Datasets
Figure 4 for Generative Models for Effective ML on Private, Decentralized Datasets
* 27 pages, 8 figures 
  Access Paper or Ask Questions

Differentially Private Learning with Adaptive Clipping


May 09, 2019
Om Thakkar, Galen Andrew, H. Brendan McMahan

Figure 1 for Differentially Private Learning with Adaptive Clipping
Figure 2 for Differentially Private Learning with Adaptive Clipping

  Access Paper or Ask Questions

SysML: The New Frontier of Machine Learning Systems


May 01, 2019
Alexander Ratner, Dan Alistarh, Gustavo Alonso, David G. Andersen, Peter Bailis, Sarah Bird, Nicholas Carlini, Bryan Catanzaro, Jennifer Chayes, Eric Chung, Bill Dally, Jeff Dean, Inderjit S. Dhillon, Alexandros Dimakis, Pradeep Dubey, Charles Elkan, Grigori Fursin, Gregory R. Ganger, Lise Getoor, Phillip B. Gibbons, Garth A. Gibson, Joseph E. Gonzalez, Justin Gottschlich, Song Han, Kim Hazelwood, Furong Huang, Martin Jaggi, Kevin Jamieson, Michael I. Jordan, Gauri Joshi, Rania Khalaf, Jason Knight, Jakub Konečný, Tim Kraska, Arun Kumar, Anastasios Kyrillidis, Aparna Lakshmiratan, Jing Li, Samuel Madden, H. Brendan McMahan, Erik Meijer, Ioannis Mitliagkas, Rajat Monga, Derek Murray, Kunle Olukotun, Dimitris Papailiopoulos, Gennady Pekhimenko, Theodoros Rekatsinas, Afshin Rostamizadeh, Christopher Ré, Christopher De Sa, Hanie Sedghi, Siddhartha Sen, Virginia Smith, Alex Smola, Dawn Song, Evan Sparks, Ion Stoica, Vivienne Sze, Madeleine Udell, Joaquin Vanschoren, Shivaram Venkataraman, Rashmi Vinayak, Markus Weimer, Andrew Gordon Wilson, Eric Xing, Matei Zaharia, Ce Zhang, Ameet Talwalkar


  Access Paper or Ask Questions

Semi-Cyclic Stochastic Gradient Descent


Apr 23, 2019
Hubert Eichner, Tomer Koren, H. Brendan McMahan, Nathan Srebro, Kunal Talwar

Figure 1 for Semi-Cyclic Stochastic Gradient Descent

  Access Paper or Ask Questions

Towards Federated Learning at Scale: System Design


Mar 22, 2019
Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konečný, Stefano Mazzocchi, H. Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, Jason Roselander

Figure 1 for Towards Federated Learning at Scale: System Design
Figure 2 for Towards Federated Learning at Scale: System Design
Figure 3 for Towards Federated Learning at Scale: System Design
Figure 4 for Towards Federated Learning at Scale: System Design

  Access Paper or Ask Questions

LEAF: A Benchmark for Federated Settings


Jan 09, 2019
Sebastian Caldas, Peter Wu, Tian Li, Jakub Konečný, H. Brendan McMahan, Virginia Smith, Ameet Talwalkar

Figure 1 for LEAF: A Benchmark for Federated Settings
Figure 2 for LEAF: A Benchmark for Federated Settings
Figure 3 for LEAF: A Benchmark for Federated Settings
Figure 4 for LEAF: A Benchmark for Federated Settings

  Access Paper or Ask Questions

Expanding the Reach of Federated Learning by Reducing Client Resource Requirements


Jan 08, 2019
Sebastian Caldas, Jakub Konečny, H. Brendan McMahan, Ameet Talwalkar

Figure 1 for Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
Figure 2 for Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
Figure 3 for Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
Figure 4 for Expanding the Reach of Federated Learning by Reducing Client Resource Requirements

  Access Paper or Ask Questions

A General Approach to Adding Differential Privacy to Iterative Training Procedures


Dec 15, 2018
H. Brendan McMahan, Galen Andrew

Figure 1 for A General Approach to Adding Differential Privacy to Iterative Training Procedures
* Presented at NeurIPS 2018 workshop on Privacy Preserving Machine Learning 
  Access Paper or Ask Questions

Learning Differentially Private Recurrent Language Models


Feb 24, 2018
H. Brendan McMahan, Daniel Ramage, Kunal Talwar, Li Zhang

Figure 1 for Learning Differentially Private Recurrent Language Models
Figure 2 for Learning Differentially Private Recurrent Language Models
Figure 3 for Learning Differentially Private Recurrent Language Models
Figure 4 for Learning Differentially Private Recurrent Language Models
* Camera-ready ICLR 2018 version, minor edits from previous 
  Access Paper or Ask Questions

Federated Learning: Strategies for Improving Communication Efficiency


Oct 30, 2017
Jakub Konečný, H. Brendan McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, Dave Bacon

Figure 1 for Federated Learning: Strategies for Improving Communication Efficiency

  Access Paper or Ask Questions

Distributed Mean Estimation with Limited Communication


Sep 25, 2017
Ananda Theertha Suresh, Felix X. Yu, Sanjiv Kumar, H. Brendan McMahan

Figure 1 for Distributed Mean Estimation with Limited Communication
Figure 2 for Distributed Mean Estimation with Limited Communication
Figure 3 for Distributed Mean Estimation with Limited Communication

  Access Paper or Ask Questions

On the Protection of Private Information in Machine Learning Systems: Two Recent Approaches


Aug 26, 2017
Martín Abadi, Úlfar Erlingsson, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Nicolas Papernot, Kunal Talwar, Li Zhang

* IEEE 30th Computer Security Foundations Symposium (CSF), pages 1--6, 2017 
  Access Paper or Ask Questions

Communication-Efficient Learning of Deep Networks from Decentralized Data


Feb 28, 2017
H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, Blaise Agüera y Arcas

Figure 1 for Communication-Efficient Learning of Deep Networks from Decentralized Data
Figure 2 for Communication-Efficient Learning of Deep Networks from Decentralized Data
Figure 3 for Communication-Efficient Learning of Deep Networks from Decentralized Data
Figure 4 for Communication-Efficient Learning of Deep Networks from Decentralized Data
* Proceedings of the 20 th International Conference on Artificial Intelligence and Statistics (AISTATS) 2017. JMLR: W&CP volume 54 
 * This version updates the large-scale LSTM experiments, along with other minor changes. In earlier versions, an inconsistency in our implementation of FedSGD caused us to report much lower learning rates for the large-scale LSTM. We reran these experiments, and also found that fewer local epochs offers better performance, leading to slightly better results for FedAvg than previously reported 
  Access Paper or Ask Questions

Practical Secure Aggregation for Federated Learning on User-Held Data


Nov 14, 2016
Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, Karn Seth

Figure 1 for Practical Secure Aggregation for Federated Learning on User-Held Data
* 5 pages, 1 figure. To appear at the NIPS 2016 workshop on Private Multi-Party Machine Learning 
  Access Paper or Ask Questions

Deep Learning with Differential Privacy


Oct 24, 2016
Martín Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, Li Zhang

Figure 1 for Deep Learning with Differential Privacy
Figure 2 for Deep Learning with Differential Privacy
Figure 3 for Deep Learning with Differential Privacy
Figure 4 for Deep Learning with Differential Privacy

  Access Paper or Ask Questions

Federated Optimization: Distributed Machine Learning for On-Device Intelligence


Oct 08, 2016
Jakub Konečný, H. Brendan McMahan, Daniel Ramage, Peter Richtárik

Figure 1 for Federated Optimization: Distributed Machine Learning for On-Device Intelligence
Figure 2 for Federated Optimization: Distributed Machine Learning for On-Device Intelligence
Figure 3 for Federated Optimization: Distributed Machine Learning for On-Device Intelligence
* 38 pages 
  Access Paper or Ask Questions

A Survey of Algorithms and Analysis for Adaptive Online Learning


Nov 09, 2015
H. Brendan McMahan

Figure 1 for A Survey of Algorithms and Analysis for Adaptive Online Learning
Figure 2 for A Survey of Algorithms and Analysis for Adaptive Online Learning
Figure 3 for A Survey of Algorithms and Analysis for Adaptive Online Learning
Figure 4 for A Survey of Algorithms and Analysis for Adaptive Online Learning

  Access Paper or Ask Questions

Unconstrained Online Linear Learning in Hilbert Spaces: Minimax Algorithms and Normal Approximations


May 21, 2014
H. Brendan McMahan, Francesco Orabona

Figure 1 for Unconstrained Online Linear Learning in Hilbert Spaces: Minimax Algorithms and Normal Approximations
* Proceedings of the 27th Annual Conference on Learning Theory (COLT 2014) 
  Access Paper or Ask Questions