Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging

Apr 30, 2020
Shigang Li, Tal Ben-Nun, Dan Alistarh, Salvatore Di Girolamo, Nikoli Dryden, Torsten Hoefler

  Access Model/Code and Paper

WoodFisher: Efficient second-order approximations for model compression

Apr 29, 2020
Sidak Pal Singh, Dan Alistarh

  Access Model/Code and Paper

Relaxed Scheduling for Scalable Belief Propagation

Feb 25, 2020
Vitaly Aksenov, Dan Alistarh, Janne H. Korhonen

  Access Model/Code and Paper

On the Sample Complexity of Adversarial Multi-Source PAC Learning

Feb 24, 2020
Nikola Konstantinov, Elias Frantar, Dan Alistarh, Christoph H. Lampert

  Access Model/Code and Paper

Distributed Mean Estimation with Optimal Error Bounds

Feb 24, 2020
Dan Alistarh, Saleh Ashkboos, Peter Davies

* 19 pages, 4 figures 

  Access Model/Code and Paper

Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent

Jan 16, 2020
Dan Alistarh, Bapi Chatterjee, Vyacheslav Kungurtsev

  Access Model/Code and Paper

PopSGD: Decentralized Stochastic Gradient Descent in the Population Model

Oct 27, 2019
Giorgi Nadiradze, Amirmojtaba Sabour, Aditya Sharma, Ilia Markov, Vitaly Aksenov, Dan Alistarh

  Access Model/Code and Paper

Powerset Convolutional Neural Networks

Oct 04, 2019
Chris Wendler, Dan Alistarh, Markus Püschel

* to appear in Proc. Neural Information Processing Systems (NeurIPS), 2019 

  Access Model/Code and Paper

Taming Unbalanced Training Workloads in Deep Learning with Partial Collective Operations

Aug 13, 2019
Shigang Li, Tal Ben-Nun, Salvatore Di Girolamo, Dan Alistarh, Torsten Hoefler

  Access Model/Code and Paper

SysML: The New Frontier of Machine Learning Systems

May 01, 2019
Alexander Ratner, Dan Alistarh, Gustavo Alonso, David G. Andersen, Peter Bailis, Sarah Bird, Nicholas Carlini, Bryan Catanzaro, Jennifer Chayes, Eric Chung, Bill Dally, Jeff Dean, Inderjit S. Dhillon, Alexandros Dimakis, Pradeep Dubey, Charles Elkan, Grigori Fursin, Gregory R. Ganger, Lise Getoor, Phillip B. Gibbons, Garth A. Gibson, Joseph E. Gonzalez, Justin Gottschlich, Song Han, Kim Hazelwood, Furong Huang, Martin Jaggi, Kevin Jamieson, Michael I. Jordan, Gauri Joshi, Rania Khalaf, Jason Knight, Jakub Konečný, Tim Kraska, Arun Kumar, Anastasios Kyrillidis, Aparna Lakshmiratan, Jing Li, Samuel Madden, H. Brendan McMahan, Erik Meijer, Ioannis Mitliagkas, Rajat Monga, Derek Murray, Kunle Olukotun, Dimitris Papailiopoulos, Gennady Pekhimenko, Theodoros Rekatsinas, Afshin Rostamizadeh, Christopher Ré, Christopher De Sa, Hanie Sedghi, Siddhartha Sen, Virginia Smith, Alex Smola, Dawn Song, Evan Sparks, Ion Stoica, Vivienne Sze, Madeleine Udell, Joaquin Vanschoren, Shivaram Venkataraman, Rashmi Vinayak, Markus Weimer, Andrew Gordon Wilson, Eric Xing, Matei Zaharia, Ce Zhang, Ameet Talwalkar

  Access Model/Code and Paper

Distributed Learning over Unreliable Networks

Oct 17, 2018
Hanlin Tang, Chen Yu, Cedric Renggli, Simon Kassing, Ankit Singla, Dan Alistarh, Ji Liu, Ce Zhang

  Access Model/Code and Paper

SparCML: High-Performance Sparse Communication for Machine Learning

Oct 02, 2018
Cédric Renggli, Dan Alistarh, Torsten Hoefler, Mehdi Aghagolzadeh

  Access Model/Code and Paper

The Convergence of Sparsified Gradient Methods

Sep 27, 2018
Dan Alistarh, Torsten Hoefler, Mikael Johansson, Sarit Khirirat, Nikola Konstantinov, Cédric Renggli

* NIPS 2018 - Advances in Neural Information Processing Systems; Authors in alphabetic order 

  Access Model/Code and Paper

The Convergence of Stochastic Gradient Descent in Asynchronous Shared Memory

Jun 22, 2018
Dan Alistarh, Christopher De Sa, Nikola Konstantinov

* To be published in PoDC 2018; 18 pages, 1 figure; Changes: added pseudocode for Algorithm 2, some references and corrected typos 

  Access Model/Code and Paper

Compressive Sensing with Low Precision Data Representation: Theory and Applications

Jun 06, 2018
Nezihe Merve Gürel, Kaan Kara, Alen Stojanov, Tyler Smith, Dan Alistarh, Markus Püschel, Ce Zhang

* 33 pages, 9 figures 

  Access Model/Code and Paper

Byzantine Stochastic Gradient Descent

Mar 23, 2018
Dan Alistarh, Zeyuan Allen-Zhu, Jerry Li

  Access Model/Code and Paper

Model compression via distillation and quantization

Feb 15, 2018
Antonio Polino, Razvan Pascanu, Dan Alistarh

* 21 pages, published as a conference paper at ICLR2018 

  Access Model/Code and Paper

DataBright: Towards a Global Exchange for Decentralized Data Ownership and Trusted Computation

Feb 13, 2018
David Dao, Dan Alistarh, Claudiu Musat, Ce Zhang

  Access Model/Code and Paper

QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding

Dec 06, 2017
Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, Milan Vojnovic

  Access Model/Code and Paper

The ZipML Framework for Training Models with End-to-End Low Precision: The Cans, the Cannots, and a Little Bit of Deep Learning

Jun 19, 2017
Hantian Zhang, Jerry Li, Kaan Kara, Dan Alistarh, Ji Liu, Ce Zhang

  Access Model/Code and Paper