Get our free extension to see links to code for papers anywhere online!

Chrome logo Add to Chrome

Firefox logo Add to Firefox

Picture for Gennady Pekhimenko

Distributed Deep Learning in Open Collaborations


Jun 18, 2021
Michael Diskin, Alexey Bukhtiyarov, Max Ryabinin, Lucile Saulnier, Quentin Lhoest, Anton Sinitsin, Dmitry Popov, Dmitry Pyrkin, Maxim Kashirin, Alexander Borzunov, Albert Villanova del Moral, Denis Mazur, Ilia Kobelev, Yacine Jernite, Thomas Wolf, Gennady Pekhimenko

* 30 pages, 9 figures. Code: https://github.com/yandex-research/DeDLOC 

  Access Paper or Ask Questions

Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices


Mar 04, 2021
Max Ryabinin, Eduard Gorbunov, Vsevolod Plokhotnyuk, Gennady Pekhimenko

* 41 pages, 6 figures 

  Access Paper or Ask Questions

RL-Scope: Cross-Stack Profiling for Deep Reinforcement Learning Workloads


Mar 04, 2021
James Gleeson, Srivatsan Krishnan, Moshe Gabel, Vijay Janapa Reddi, Eyal de Lara, Gennady Pekhimenko

* RL-Scope is an open-source tool available at https://github.com/UofT-EcoSystem/rlscope . Proceedings of the 4th MLSys Conference, 2021. Changes: camera ready for MLSys publication -- shorten abstract, add acknowledgements, minor grammar fixes 

  Access Paper or Ask Questions

Horizontally Fused Training Array: An Effective Hardware Utilization Squeezer for Training Novel Deep Learning Models


Feb 07, 2021
Shang Wang, Peiming Yang, Yuxuan Zheng, Xin Li, Gennady Pekhimenko

* Submission draft (as a preview) to MLSys 2021 

  Access Paper or Ask Questions

Computational Performance Predictions for Deep Neural Network Training: A Runtime-Based Approach


Jan 31, 2021
Geoffrey X. Yu, Yubo Gao, Pavel Golikov, Gennady Pekhimenko

* 17 pages, 7 figures 

  Access Paper or Ask Questions

IOS: Inter-Operator Scheduler for CNN Acceleration


Nov 02, 2020
Yaoyao Ding, Ligeng Zhu, Zhihao Jia, Gennady Pekhimenko, Song Han


  Access Paper or Ask Questions

FPRaker: A Processing Element For Accelerating Neural Network Training


Oct 15, 2020
Omar Mohamed Awad, Mostafa Mahmoud, Isak Edo, Ali Hadi Zadeh, Ciaran Bannon, Anand Jayarajan, Gennady Pekhimenko, Andreas Moshovos


  Access Paper or Ask Questions

TensorDash: Exploiting Sparsity to Accelerate Deep Neural Network Training and Inference


Sep 01, 2020
Mostafa Mahmoud, Isak Edo, Ali Hadi Zadeh, Omar Mohamed Awad, Gennady Pekhimenko, Jorge Albericio, Andreas Moshovos


  Access Paper or Ask Questions

Skyline: Interactive In-Editor Computational Performance Profiling for Deep Neural Network Training


Aug 20, 2020
Geoffrey X. Yu, Tovi Grossman, Gennady Pekhimenko

* 14 pages, 5 figures. Appears in the proceedings of UIST'20 

  Access Paper or Ask Questions

Multi-node Bert-pretraining: Cost-efficient Approach


Aug 01, 2020
Jiahuang Lin, Xin Li, Gennady Pekhimenko


  Access Paper or Ask Questions

Daydream: Accurately Estimating the Efficacy of Optimizations for DNN Training


Jun 05, 2020
Hongyu Zhu, Amar Phanishayee, Gennady Pekhimenko


  Access Paper or Ask Questions

MLPerf Inference Benchmark


Nov 06, 2019
Vijay Janapa Reddi, Christine Cheng, David Kanter, Peter Mattson, Guenther Schmuelling, Carole-Jean Wu, Brian Anderson, Maximilien Breughe, Mark Charlebois, William Chou, Ramesh Chukka, Cody Coleman, Sam Davis, Pan Deng, Greg Diamos, Jared Duke, Dave Fick, J. Scott Gardner, Itay Hubara, Sachin Idgunji, Thomas B. Jablin, Jeff Jiao, Tom St. John, Pankaj Kanwar, David Lee, Jeffery Liao, Anton Lokhmotov, Francisco Massa, Peng Meng, Paulius Micikevicius, Colin Osborne, Gennady Pekhimenko, Arun Tejusve Raghunath Rajan, Dilip Sequeira, Ashish Sirasao, Fei Sun, Hanlin Tang, Michael Thomson, Frank Wei, Ephrem Wu, Lingjie Xu, Koichi Yamada, Bing Yu, George Yuan, Aaron Zhong, Peizhao Zhang, Yuchen Zhou


  Access Paper or Ask Questions

MLPerf Training Benchmark


Oct 30, 2019
Peter Mattson, Christine Cheng, Cody Coleman, Greg Diamos, Paulius Micikevicius, David Patterson, Hanlin Tang, Gu-Yeon Wei, Peter Bailis, Victor Bittorf, David Brooks, Dehao Chen, Debojyoti Dutta, Udit Gupta, Kim Hazelwood, Andrew Hock, Xinyuan Huang, Bill Jia, Daniel Kang, David Kanter, Naveen Kumar, Jeffery Liao, Guokai Ma, Deepak Narayanan, Tayo Oguntebi, Gennady Pekhimenko, Lillian Pentecost, Vijay Janapa Reddi, Taylor Robie, Tom St. John, Carole-Jean Wu, Lingjie Xu, Cliff Young, Matei Zaharia


  Access Paper or Ask Questions

Scaling Back-propagation by Parallel Scan Algorithm


Jul 23, 2019
Shang Wang, Yifan Bai, Gennady Pekhimenko


  Access Paper or Ask Questions

Priority-based Parameter Propagation for Distributed DNN Training


May 10, 2019
Anand Jayarajan, Jinliang Wei, Garth Gibson, Alexandra Fedorova, Gennady Pekhimenko

* In proceedings of the 2nd SysML Conference 2019 

  Access Paper or Ask Questions

SysML: The New Frontier of Machine Learning Systems


May 01, 2019
Alexander Ratner, Dan Alistarh, Gustavo Alonso, David G. Andersen, Peter Bailis, Sarah Bird, Nicholas Carlini, Bryan Catanzaro, Jennifer Chayes, Eric Chung, Bill Dally, Jeff Dean, Inderjit S. Dhillon, Alexandros Dimakis, Pradeep Dubey, Charles Elkan, Grigori Fursin, Gregory R. Ganger, Lise Getoor, Phillip B. Gibbons, Garth A. Gibson, Joseph E. Gonzalez, Justin Gottschlich, Song Han, Kim Hazelwood, Furong Huang, Martin Jaggi, Kevin Jamieson, Michael I. Jordan, Gauri Joshi, Rania Khalaf, Jason Knight, Jakub Konečný, Tim Kraska, Arun Kumar, Anastasios Kyrillidis, Aparna Lakshmiratan, Jing Li, Samuel Madden, H. Brendan McMahan, Erik Meijer, Ioannis Mitliagkas, Rajat Monga, Derek Murray, Kunle Olukotun, Dimitris Papailiopoulos, Gennady Pekhimenko, Theodoros Rekatsinas, Afshin Rostamizadeh, Christopher Ré, Christopher De Sa, Hanie Sedghi, Siddhartha Sen, Virginia Smith, Alex Smola, Dawn Song, Evan Sparks, Ion Stoica, Vivienne Sze, Madeleine Udell, Joaquin Vanschoren, Shivaram Venkataraman, Rashmi Vinayak, Markus Weimer, Andrew Gordon Wilson, Eric Xing, Matei Zaharia, Ce Zhang, Ameet Talwalkar


  Access Paper or Ask Questions

EcoRNN: Fused LSTM RNN Implementation with Data Layout Optimization


May 22, 2018
Bojian Zheng, Akshay Nair, Qiongsi Wu, Nandita Vijaykumar, Gennady Pekhimenko


  Access Paper or Ask Questions

TBD: Benchmarking and Analyzing Deep Neural Network Training


Apr 14, 2018
Hongyu Zhu, Mohamed Akrout, Bojian Zheng, Andrew Pelegris, Amar Phanishayee, Bianca Schroeder, Gennady Pekhimenko


  Access Paper or Ask Questions