Picture for Pieter Noordhuis

Pieter Noordhuis

PyTorch Distributed: Experiences on Accelerating Data Parallel Training

Jun 28, 2020
Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, Soumith Chintala

Figure 1 for PyTorch Distributed: Experiences on Accelerating Data Parallel Training

Figure 2 for PyTorch Distributed: Experiences on Accelerating Data Parallel Training

Figure 3 for PyTorch Distributed: Experiences on Accelerating Data Parallel Training

Figure 4 for PyTorch Distributed: Experiences on Accelerating Data Parallel Training

* To appear in VLDB 2020

Access Paper or Ask Questions

Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour

Apr 30, 2018
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, Kaiming He

Figure 1 for Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour

Figure 2 for Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour

Figure 3 for Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour

Figure 4 for Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour

* Tech report (v2: correct typos)

Access Paper or Ask Questions