Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Torsten Hoefler

Torsten Hoefler

ETH Zürich

Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines

Jul 14, 2021
Shigang Li, Torsten Hoefler

* The paper was accepted by the 2021 International Conference for High Performance Computing, Networking, Storage and Analysis (SC'21), in Best Paper Finalist 

  Access Paper or Ask Questions

Learning Combinatorial Node Labeling Algorithms

Jun 15, 2021
Lukas Gianinazzi, Maximilian Fries, Nikoli Dryden, Tal Ben-Nun, Maciej Besta, Torsten Hoefler

  Access Paper or Ask Questions

Motif Prediction with Graph Neural Networks

Jun 05, 2021
Maciej Besta, Raphael Grob, Cesare Miglioli, Nicola Bernold, Grzegorz Kwasniewski, Gabriel Gjini, Raghavendra Kanakagiri, Saleh Ashkboos, Lukas Gianinazzi, Nikoli Dryden, Torsten Hoefler

  Access Paper or Ask Questions

GraphMineSuite: Enabling High-Performance and Programmable Graph Mining Algorithms with Set Algebra

Mar 05, 2021
Maciej Besta, Zur Vonarburg-Shmaria, Yannick Schaffner, Leonardo Schwarz, Grzegorz Kwasniewski, Lukas Gianinazzi, Jakub Beranek, Kacper Janda, Tobias Holenstein, Sebastian Leisinger, Peter Tatkowski, Esref Ozdemir, Adrian Balla, Marcin Copik, Philipp Lindenberger, Pavel Kalvoda, Marek Konieczny, Onur Mutlu, Torsten Hoefler

  Access Paper or Ask Questions

Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks

Jan 31, 2021
Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, Alexandra Peste

* 90 pages, 26 figures 

  Access Paper or Ask Questions

Clairvoyant Prefetching for Distributed Machine Learning I/O

Jan 21, 2021
Roman Böhringer, Nikoli Dryden, Tal Ben-Nun, Torsten Hoefler

* 15 pages, 11 figures 

  Access Paper or Ask Questions

Deep Data Flow Analysis

Nov 21, 2020
Chris Cummins, Hugh Leather, Zacharias Fisches, Tal Ben-Nun, Torsten Hoefler, Michael O'Boyle

* 9 pages, plus appendices. arXiv admin note: text overlap with arXiv:2003.10536 

  Access Paper or Ask Questions

Data Movement Is All You Need: A Case Study on Optimizing Transformers

Jul 02, 2020
Andrei Ivanov, Nikoli Dryden, Tal Ben-Nun, Shigang Li, Torsten Hoefler

* 15 pages, 6 figures; minor clarifications and style updates 

  Access Paper or Ask Questions

Shapeshifter Networks: Cross-layer Parameter Sharing for Scalable and Effective Deep Learning

Jun 18, 2020
Bryan A. Plummer, Nikoli Dryden, Julius Frost, Torsten Hoefler, Kate Saenko

  Access Paper or Ask Questions

Deep Learning for Post-Processing Ensemble Weather Forecasts

May 18, 2020
Peter Grönquist, Chengyuan Yao, Tal Ben-Nun, Nikoli Dryden, Peter Dueben, Shigang Li, Torsten Hoefler

  Access Paper or Ask Questions

Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging

Apr 30, 2020
Shigang Li, Tal Ben-Nun, Dan Alistarh, Salvatore Di Girolamo, Nikoli Dryden, Torsten Hoefler

  Access Paper or Ask Questions

ProGraML: Graph-based Deep Learning for Program Optimization and Analysis

Mar 23, 2020
Chris Cummins, Zacharias V. Fisches, Tal Ben-Nun, Torsten Hoefler, Hugh Leather

* 20 pages, author preprint 

  Access Paper or Ask Questions

Predicting Weather Uncertainty with Deep Convnets

Dec 04, 2019
Peter Grönquist, Tal Ben-Nun, Nikoli Dryden, Peter Dueben, Luca Lavarini, Shigang Li, Torsten Hoefler

* Poster presentation at NeurIPS2019 "Machine Learning and the Physical Sciences" Workshop 

  Access Paper or Ask Questions

Taming Unbalanced Training Workloads in Deep Learning with Partial Collective Operations

Aug 13, 2019
Shigang Li, Tal Ben-Nun, Salvatore Di Girolamo, Dan Alistarh, Torsten Hoefler

  Access Paper or Ask Questions

Mix & Match: training convnets with mixed image sizes for improved accuracy, speed and scale resiliency

Aug 12, 2019
Elad Hoffer, Berry Weinstein, Itay Hubara, Tal Ben-Nun, Torsten Hoefler, Daniel Soudry

  Access Paper or Ask Questions

A Modular Benchmarking Infrastructure for High-Performance and Reproducible Deep Learning

Jan 29, 2019
Tal Ben-Nun, Maciej Besta, Simon Huber, Alexandros Nikolaos Ziogas, Daniel Peter, Torsten Hoefler

* Accepted to IPDPS 2019 

  Access Paper or Ask Questions

Augment your batch: better training with larger batches

Jan 27, 2019
Elad Hoffer, Tal Ben-Nun, Itay Hubara, Niv Giladi, Torsten Hoefler, Daniel Soudry

  Access Paper or Ask Questions

SparCML: High-Performance Sparse Communication for Machine Learning

Oct 02, 2018
Cédric Renggli, Dan Alistarh, Torsten Hoefler, Mehdi Aghagolzadeh

  Access Paper or Ask Questions

The Convergence of Sparsified Gradient Methods

Sep 27, 2018
Dan Alistarh, Torsten Hoefler, Mikael Johansson, Sarit Khirirat, Nikola Konstantinov, Cédric Renggli

* NIPS 2018 - Advances in Neural Information Processing Systems; Authors in alphabetic order 

  Access Paper or Ask Questions

Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis

Sep 15, 2018
Tal Ben-Nun, Torsten Hoefler

  Access Paper or Ask Questions

Neural Code Comprehension: A Learnable Representation of Code Semantics

Jul 31, 2018
Tal Ben-Nun, Alice Shoshana Jakobovits, Torsten Hoefler

  Access Paper or Ask Questions

μ-cuDNN: Accelerating Deep Learning Frameworks with Micro-Batching

Apr 13, 2018
Yosuke Oyama, Tal Ben-Nun, Torsten Hoefler, Satoshi Matsuoka

* 11 pages, 14 figures. Part of the content have been published in IPSJ SIG Technical Report, Vol. 2017-HPC-162, No. 22, pp. 1-9, 2017. (DOI:

  Access Paper or Ask Questions