Alert button
Picture for Guojing Cong

Guojing Cong

Alert button

Optimizing Distributed Training on Frontier for Large Language Models

Dec 21, 2023
Sajal Dash, Isaac Lyngaas, Junqi Yin, Xiao Wang, Romain Egele, Guojing Cong, Feiyi Wang, Prasanna Balaprakash

Viaarxiv icon

DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies

Oct 11, 2023
Shuaiwen Leon Song, Bonnie Kruft, Minjia Zhang, Conglong Li, Shiyang Chen, Chengming Zhang, Masahiro Tanaka, Xiaoxia Wu, Jeff Rasley, Ammar Ahmad Awan, Connor Holmes, Martin Cai, Adam Ghanem, Zhongzhu Zhou, Yuxiong He, Pete Luferenko, Divya Kumar, Jonathan Weyn, Ruixiong Zhang, Sylwester Klocek, Volodymyr Vragov, Mohammed AlQuraishi, Gustaf Ahdritz, Christina Floristean, Cristina Negri, Rao Kotamarthi, Venkatram Vishwanath, Arvind Ramanathan, Sam Foreman, Kyle Hippe, Troy Arcomano, Romit Maulik, Maxim Zvyagin, Alexander Brace, Bin Zhang, Cindy Orozco Bohorquez, Austin Clyde, Bharat Kale, Danilo Perez-Rivera, Heng Ma, Carla M. Mann, Michael Irvin, J. Gregory Pauloski, Logan Ward, Valerie Hayot, Murali Emani, Zhen Xie, Diangen Lin, Maulik Shukla, Ian Foster, James J. Davis, Michael E. Papka, Thomas Brettin, Prasanna Balaprakash, Gina Tourassi, John Gounley, Heidi Hanson, Thomas E Potok, Massimiliano Lupo Pasini, Kate Evans, Dan Lu, Dalton Lunga, Junqi Yin, Sajal Dash, Feiyi Wang, Mallikarjun Shankar, Isaac Lyngaas, Xiao Wang, Guojing Cong, Pei Zhang, Ming Fan, Siyan Liu, Adolfy Hoisie, Shinjae Yoo, Yihui Ren, William Tang, Kyle Felker, Alexey Svyatkovskiy, Hang Liu, Ashwin Aji, Angela Dalton, Michael Schulte, Karl Schulz, Yuntian Deng, Weili Nie, Josh Romero, Christian Dallago, Arash Vahdat, Chaowei Xiao, Thomas Gibbs, Anima Anandkumar, Rick Stevens

Figure 1 for DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
Figure 2 for DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
Figure 3 for DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
Figure 4 for DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
Viaarxiv icon

AI-aided multiscale modeling of physiologically-significant blood clots

May 25, 2022
Yicong Zhu, Changnian Han, Peng Zhang, Guojing Cong, James R. Kozloski, Chih-Chieh Yang, Leili Zhang, Yuefan Deng

Viaarxiv icon

Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum

Oct 01, 2021
Guojing Cong, Tianyi Liu

Figure 1 for Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum
Figure 2 for Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum
Figure 3 for Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum
Figure 4 for Accelerate Distributed Stochastic Descent for Nonconvex Optimization with Momentum
Viaarxiv icon

CASTELO: Clustered Atom Subtypes aidEd Lead Optimization -- a combined machine learning and molecular modeling method

Nov 27, 2020
Leili Zhang, Giacomo Domeniconi, Chih-Chieh Yang, Seung-gu Kang, Ruhong Zhou, Guojing Cong

Figure 1 for CASTELO: Clustered Atom Subtypes aidEd Lead Optimization -- a combined machine learning and molecular modeling method
Figure 2 for CASTELO: Clustered Atom Subtypes aidEd Lead Optimization -- a combined machine learning and molecular modeling method
Figure 3 for CASTELO: Clustered Atom Subtypes aidEd Lead Optimization -- a combined machine learning and molecular modeling method
Figure 4 for CASTELO: Clustered Atom Subtypes aidEd Lead Optimization -- a combined machine learning and molecular modeling method
Viaarxiv icon

Accelerating Data Loading in Deep Neural Network Training

Oct 02, 2019
Chih-Chieh Yang, Guojing Cong

Figure 1 for Accelerating Data Loading in Deep Neural Network Training
Figure 2 for Accelerating Data Loading in Deep Neural Network Training
Figure 3 for Accelerating Data Loading in Deep Neural Network Training
Figure 4 for Accelerating Data Loading in Deep Neural Network Training
Viaarxiv icon

A Distributed Hierarchical SGD Algorithm with Sparse Global Reduction

Mar 12, 2019
Fan Zhou, Guojing Cong

Figure 1 for A Distributed Hierarchical SGD Algorithm with Sparse Global Reduction
Figure 2 for A Distributed Hierarchical SGD Algorithm with Sparse Global Reduction
Figure 3 for A Distributed Hierarchical SGD Algorithm with Sparse Global Reduction
Figure 4 for A Distributed Hierarchical SGD Algorithm with Sparse Global Reduction
Viaarxiv icon

On the convergence properties of a $K$-step averaging stochastic gradient descent algorithm for nonconvex optimization

May 16, 2018
Fan Zhou, Guojing Cong

Figure 1 for On the convergence properties of a $K$-step averaging stochastic gradient descent algorithm for nonconvex optimization
Figure 2 for On the convergence properties of a $K$-step averaging stochastic gradient descent algorithm for nonconvex optimization
Figure 3 for On the convergence properties of a $K$-step averaging stochastic gradient descent algorithm for nonconvex optimization
Figure 4 for On the convergence properties of a $K$-step averaging stochastic gradient descent algorithm for nonconvex optimization
Viaarxiv icon