Get our free extension to see links to code for papers anywhere online!

Add to Chrome

Add to Firefox

Picture for Igor Gitman

Igor Gitman

Understanding the Role of Momentum in Stochastic Gradient Methods

Oct 30, 2019
Igor Gitman, Hunter Lang, Pengchuan Zhang, Lin Xiao

Figure 1 for Understanding the Role of Momentum in Stochastic Gradient Methods

Figure 2 for Understanding the Role of Momentum in Stochastic Gradient Methods

Figure 3 for Understanding the Role of Momentum in Stochastic Gradient Methods

Figure 4 for Understanding the Role of Momentum in Stochastic Gradient Methods

* 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada

Access Paper or Ask Questions

OpenSeq2Seq: extensible toolkit for distributed and mixed precision training of sequence-to-sequence models

May 25, 2018
Oleksii Kuchaiev, Boris Ginsburg, Igor Gitman, Vitaly Lavrukhin, Carl Case, Paulius Micikevicius

Figure 1 for OpenSeq2Seq: extensible toolkit for distributed and mixed precision training of sequence-to-sequence models

Figure 2 for OpenSeq2Seq: extensible toolkit for distributed and mixed precision training of sequence-to-sequence models

Figure 3 for OpenSeq2Seq: extensible toolkit for distributed and mixed precision training of sequence-to-sequence models

Figure 4 for OpenSeq2Seq: extensible toolkit for distributed and mixed precision training of sequence-to-sequence models

* to be presented at Workshop for Natural Language Processing Open Source Software (NLP-OSS), co-located with ACL2018

Access Paper or Ask Questions

Convergence Analysis of Gradient Descent Algorithms with Proportional Updates

Jan 09, 2018
Igor Gitman, Deepak Dilipkumar, Ben Parr

Figure 1 for Convergence Analysis of Gradient Descent Algorithms with Proportional Updates

Figure 2 for Convergence Analysis of Gradient Descent Algorithms with Proportional Updates

Figure 3 for Convergence Analysis of Gradient Descent Algorithms with Proportional Updates

* Source code (uses TensorFlow): https://github.com/bparr/lars

Access Paper or Ask Questions

Comparison of Batch Normalization and Weight Normalization Algorithms for the Large-scale Image Classification

Oct 07, 2017
Igor Gitman, Boris Ginsburg

Figure 1 for Comparison of Batch Normalization and Weight Normalization Algorithms for the Large-scale Image Classification

Figure 2 for Comparison of Batch Normalization and Weight Normalization Algorithms for the Large-scale Image Classification

Figure 3 for Comparison of Batch Normalization and Weight Normalization Algorithms for the Large-scale Image Classification

Figure 4 for Comparison of Batch Normalization and Weight Normalization Algorithms for the Large-scale Image Classification

Access Paper or Ask Questions