Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox
Picture for Dan Alistarh

Dan Alistarh

NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization


May 01, 2021
Ali Ramezani-Kebrya, Fartash Faghri, Ilya Markov, Vitalii Aksenov, Dan Alistarh, Daniel M. Roy

Figure 1 for NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization
Figure 2 for NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization
Figure 3 for NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization
Figure 4 for NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization
* This entry is redundant and was created in error. See arXiv:1908.06077 for the latest version 
  Access Paper or Ask Questions

Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks


Jan 31, 2021
Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, Alexandra Peste

Figure 1 for Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
Figure 2 for Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
Figure 3 for Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
Figure 4 for Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
* 90 pages, 26 figures 
  Access Paper or Ask Questions

Byzantine-Resilient Non-Convex Stochastic Gradient Descent


Dec 28, 2020
Zeyuan Allen-Zhu, Faeze Ebrahimian, Jerry Li, Dan Alistarh

Figure 1 for Byzantine-Resilient Non-Convex Stochastic Gradient Descent
Figure 2 for Byzantine-Resilient Non-Convex Stochastic Gradient Descent
Figure 3 for Byzantine-Resilient Non-Convex Stochastic Gradient Descent
Figure 4 for Byzantine-Resilient Non-Convex Stochastic Gradient Descent

  Access Paper or Ask Questions

Adaptive Gradient Quantization for Data-Parallel SGD


Oct 23, 2020
Fartash Faghri, Iman Tabrizian, Ilia Markov, Dan Alistarh, Daniel Roy, Ali Ramezani-Kebrya

Figure 1 for Adaptive Gradient Quantization for Data-Parallel SGD
Figure 2 for Adaptive Gradient Quantization for Data-Parallel SGD
Figure 3 for Adaptive Gradient Quantization for Data-Parallel SGD
Figure 4 for Adaptive Gradient Quantization for Data-Parallel SGD
* Accepted at the conference on Neural Information Processing Systems (NeurIPS 2020) 
  Access Paper or Ask Questions

Improved Communication Lower Bounds for Distributed Optimisation


Oct 16, 2020
Dan Alistarh, Janne H. Korhonen

Figure 1 for Improved Communication Lower Bounds for Distributed Optimisation
Figure 2 for Improved Communication Lower Bounds for Distributed Optimisation

  Access Paper or Ask Questions

Stochastic Gradient Langevin with Delayed Gradients


Jun 12, 2020
Vyacheslav Kungurtsev, Bapi Chatterjee, Dan Alistarh

Figure 1 for Stochastic Gradient Langevin with Delayed Gradients
Figure 2 for Stochastic Gradient Langevin with Delayed Gradients
Figure 3 for Stochastic Gradient Langevin with Delayed Gradients
Figure 4 for Stochastic Gradient Langevin with Delayed Gradients

  Access Paper or Ask Questions

Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging


Apr 30, 2020
Shigang Li, Tal Ben-Nun, Dan Alistarh, Salvatore Di Girolamo, Nikoli Dryden, Torsten Hoefler

Figure 1 for Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging
Figure 2 for Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging
Figure 3 for Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging
Figure 4 for Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging

  Access Paper or Ask Questions

WoodFisher: Efficient second-order approximations for model compression


Apr 29, 2020
Sidak Pal Singh, Dan Alistarh

Figure 1 for WoodFisher: Efficient second-order approximations for model compression
Figure 2 for WoodFisher: Efficient second-order approximations for model compression
Figure 3 for WoodFisher: Efficient second-order approximations for model compression
Figure 4 for WoodFisher: Efficient second-order approximations for model compression

  Access Paper or Ask Questions

Relaxed Scheduling for Scalable Belief Propagation


Feb 25, 2020
Vitaly Aksenov, Dan Alistarh, Janne H. Korhonen

Figure 1 for Relaxed Scheduling for Scalable Belief Propagation
Figure 2 for Relaxed Scheduling for Scalable Belief Propagation
Figure 3 for Relaxed Scheduling for Scalable Belief Propagation
Figure 4 for Relaxed Scheduling for Scalable Belief Propagation

  Access Paper or Ask Questions

On the Sample Complexity of Adversarial Multi-Source PAC Learning


Feb 24, 2020
Nikola Konstantinov, Elias Frantar, Dan Alistarh, Christoph H. Lampert


  Access Paper or Ask Questions

Distributed Mean Estimation with Optimal Error Bounds


Feb 24, 2020
Dan Alistarh, Saleh Ashkboos, Peter Davies

Figure 1 for Distributed Mean Estimation with Optimal Error Bounds
Figure 2 for Distributed Mean Estimation with Optimal Error Bounds
Figure 3 for Distributed Mean Estimation with Optimal Error Bounds
Figure 4 for Distributed Mean Estimation with Optimal Error Bounds
* 19 pages, 4 figures 
  Access Paper or Ask Questions

Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent


Jan 16, 2020
Dan Alistarh, Bapi Chatterjee, Vyacheslav Kungurtsev

Figure 1 for Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent
Figure 2 for Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent
Figure 3 for Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent
Figure 4 for Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent

  Access Paper or Ask Questions

PopSGD: Decentralized Stochastic Gradient Descent in the Population Model


Oct 27, 2019
Giorgi Nadiradze, Amirmojtaba Sabour, Aditya Sharma, Ilia Markov, Vitaly Aksenov, Dan Alistarh

Figure 1 for PopSGD: Decentralized Stochastic Gradient Descent in the Population Model
Figure 2 for PopSGD: Decentralized Stochastic Gradient Descent in the Population Model
Figure 3 for PopSGD: Decentralized Stochastic Gradient Descent in the Population Model
Figure 4 for PopSGD: Decentralized Stochastic Gradient Descent in the Population Model

  Access Paper or Ask Questions

Powerset Convolutional Neural Networks


Oct 04, 2019
Chris Wendler, Dan Alistarh, Markus Püschel

Figure 1 for Powerset Convolutional Neural Networks
Figure 2 for Powerset Convolutional Neural Networks
Figure 3 for Powerset Convolutional Neural Networks
* to appear in Proc. Neural Information Processing Systems (NeurIPS), 2019 
  Access Paper or Ask Questions

Taming Unbalanced Training Workloads in Deep Learning with Partial Collective Operations


Aug 13, 2019
Shigang Li, Tal Ben-Nun, Salvatore Di Girolamo, Dan Alistarh, Torsten Hoefler

Figure 1 for Taming Unbalanced Training Workloads in Deep Learning with Partial Collective Operations
Figure 2 for Taming Unbalanced Training Workloads in Deep Learning with Partial Collective Operations
Figure 3 for Taming Unbalanced Training Workloads in Deep Learning with Partial Collective Operations
Figure 4 for Taming Unbalanced Training Workloads in Deep Learning with Partial Collective Operations

  Access Paper or Ask Questions

SysML: The New Frontier of Machine Learning Systems


May 01, 2019
Alexander Ratner, Dan Alistarh, Gustavo Alonso, David G. Andersen, Peter Bailis, Sarah Bird, Nicholas Carlini, Bryan Catanzaro, Jennifer Chayes, Eric Chung, Bill Dally, Jeff Dean, Inderjit S. Dhillon, Alexandros Dimakis, Pradeep Dubey, Charles Elkan, Grigori Fursin, Gregory R. Ganger, Lise Getoor, Phillip B. Gibbons, Garth A. Gibson, Joseph E. Gonzalez, Justin Gottschlich, Song Han, Kim Hazelwood, Furong Huang, Martin Jaggi, Kevin Jamieson, Michael I. Jordan, Gauri Joshi, Rania Khalaf, Jason Knight, Jakub Konečný, Tim Kraska, Arun Kumar, Anastasios Kyrillidis, Aparna Lakshmiratan, Jing Li, Samuel Madden, H. Brendan McMahan, Erik Meijer, Ioannis Mitliagkas, Rajat Monga, Derek Murray, Kunle Olukotun, Dimitris Papailiopoulos, Gennady Pekhimenko, Theodoros Rekatsinas, Afshin Rostamizadeh, Christopher Ré, Christopher De Sa, Hanie Sedghi, Siddhartha Sen, Virginia Smith, Alex Smola, Dawn Song, Evan Sparks, Ion Stoica, Vivienne Sze, Madeleine Udell, Joaquin Vanschoren, Shivaram Venkataraman, Rashmi Vinayak, Markus Weimer, Andrew Gordon Wilson, Eric Xing, Matei Zaharia, Ce Zhang, Ameet Talwalkar


  Access Paper or Ask Questions

Distributed Learning over Unreliable Networks


Oct 17, 2018
Hanlin Tang, Chen Yu, Cedric Renggli, Simon Kassing, Ankit Singla, Dan Alistarh, Ji Liu, Ce Zhang

Figure 1 for Distributed Learning over Unreliable Networks
Figure 2 for Distributed Learning over Unreliable Networks
Figure 3 for Distributed Learning over Unreliable Networks
Figure 4 for Distributed Learning over Unreliable Networks

  Access Paper or Ask Questions

SparCML: High-Performance Sparse Communication for Machine Learning


Oct 02, 2018
Cédric Renggli, Dan Alistarh, Torsten Hoefler, Mehdi Aghagolzadeh

Figure 1 for SparCML: High-Performance Sparse Communication for Machine Learning
Figure 2 for SparCML: High-Performance Sparse Communication for Machine Learning
Figure 3 for SparCML: High-Performance Sparse Communication for Machine Learning
Figure 4 for SparCML: High-Performance Sparse Communication for Machine Learning

  Access Paper or Ask Questions

The Convergence of Sparsified Gradient Methods


Sep 27, 2018
Dan Alistarh, Torsten Hoefler, Mikael Johansson, Sarit Khirirat, Nikola Konstantinov, Cédric Renggli

Figure 1 for The Convergence of Sparsified Gradient Methods
Figure 2 for The Convergence of Sparsified Gradient Methods
Figure 3 for The Convergence of Sparsified Gradient Methods
* NIPS 2018 - Advances in Neural Information Processing Systems; Authors in alphabetic order 
  Access Paper or Ask Questions

The Convergence of Stochastic Gradient Descent in Asynchronous Shared Memory


Jun 22, 2018
Dan Alistarh, Christopher De Sa, Nikola Konstantinov

Figure 1 for The Convergence of Stochastic Gradient Descent in Asynchronous Shared Memory
* To be published in PoDC 2018; 18 pages, 1 figure; Changes: added pseudocode for Algorithm 2, some references and corrected typos 
  Access Paper or Ask Questions

Compressive Sensing with Low Precision Data Representation: Theory and Applications


Jun 06, 2018
Nezihe Merve Gürel, Kaan Kara, Alen Stojanov, Tyler Smith, Dan Alistarh, Markus Püschel, Ce Zhang

Figure 1 for Compressive Sensing with Low Precision Data Representation: Theory and Applications
Figure 2 for Compressive Sensing with Low Precision Data Representation: Theory and Applications
Figure 3 for Compressive Sensing with Low Precision Data Representation: Theory and Applications
Figure 4 for Compressive Sensing with Low Precision Data Representation: Theory and Applications
* 33 pages, 9 figures 
  Access Paper or Ask Questions

Byzantine Stochastic Gradient Descent


Mar 23, 2018
Dan Alistarh, Zeyuan Allen-Zhu, Jerry Li

Figure 1 for Byzantine Stochastic Gradient Descent

  Access Paper or Ask Questions

Model compression via distillation and quantization


Feb 15, 2018
Antonio Polino, Razvan Pascanu, Dan Alistarh

Figure 1 for Model compression via distillation and quantization
Figure 2 for Model compression via distillation and quantization
Figure 3 for Model compression via distillation and quantization
* 21 pages, published as a conference paper at ICLR2018 
  Access Paper or Ask Questions

DataBright: Towards a Global Exchange for Decentralized Data Ownership and Trusted Computation


Feb 13, 2018
David Dao, Dan Alistarh, Claudiu Musat, Ce Zhang


  Access Paper or Ask Questions

QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding


Dec 06, 2017
Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, Milan Vojnovic

Figure 1 for QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
Figure 2 for QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
Figure 3 for QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding
Figure 4 for QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding

  Access Paper or Ask Questions

The ZipML Framework for Training Models with End-to-End Low Precision: The Cans, the Cannots, and a Little Bit of Deep Learning


Jun 19, 2017
Hantian Zhang, Jerry Li, Kaan Kara, Dan Alistarh, Ji Liu, Ce Zhang

Figure 1 for The ZipML Framework for Training Models with End-to-End Low Precision: The Cans, the Cannots, and a Little Bit of Deep Learning
Figure 2 for The ZipML Framework for Training Models with End-to-End Low Precision: The Cans, the Cannots, and a Little Bit of Deep Learning
Figure 3 for The ZipML Framework for Training Models with End-to-End Low Precision: The Cans, the Cannots, and a Little Bit of Deep Learning
Figure 4 for The ZipML Framework for Training Models with End-to-End Low Precision: The Cans, the Cannots, and a Little Bit of Deep Learning

  Access Paper or Ask Questions