Get our free extension to see links to code for papers anywhere online!

Chrome logo  Add to Chrome

Firefox logo Add to Firefox
Picture for Yanli Zhao

Yanli Zhao
Alert button

PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel


Apr 21, 2023
Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, Alban Desmaison, Can Balioglu, Bernard Nguyen, Geeta Chauhan, Yuchen Hao, Shen Li

Add code
Alert button

Figure 1 for PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
Figure 2 for PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
Figure 3 for PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
Figure 4 for PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel

   Access Paper or Ask Questions
  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email

PyTorch Distributed: Experiences on Accelerating Data Parallel Training


Jun 28, 2020
Shen Li, Yanli Zhao, Rohan Varma, Omkar Salpekar, Pieter Noordhuis, Teng Li, Adam Paszke, Jeff Smith, Brian Vaughan, Pritam Damania, Soumith Chintala

Add code
Alert button

Figure 1 for PyTorch Distributed: Experiences on Accelerating Data Parallel Training
Figure 2 for PyTorch Distributed: Experiences on Accelerating Data Parallel Training
Figure 3 for PyTorch Distributed: Experiences on Accelerating Data Parallel Training
Figure 4 for PyTorch Distributed: Experiences on Accelerating Data Parallel Training
* To appear in VLDB 2020 
   Access Paper or Ask Questions
  • Share via Twitter
  • Share via Facebook
  • Share via LinkedIn
  • Share via Whatsapp
  • Share via Messenger
  • Share via Email