Picture for Can Karakus

Can Karakus

Amazon SageMaker Model Parallelism: A General and Flexible Framework for Large Model Training

Nov 10, 2021
Can Karakus, Rahul Huilgol, Fei Wu, Anirudh Subramanian, Cade Daniel, Derya Cavdar, Teng Xu, Haohan Chen, Arash Rahnama, Luis Quintela

Figure 1 for Amazon SageMaker Model Parallelism: A General and Flexible Framework for Large Model Training

Figure 2 for Amazon SageMaker Model Parallelism: A General and Flexible Framework for Large Model Training

Figure 3 for Amazon SageMaker Model Parallelism: A General and Flexible Framework for Large Model Training

Figure 4 for Amazon SageMaker Model Parallelism: A General and Flexible Framework for Large Model Training

* 24 pages. Submitted for review

Access Paper or Ask Questions

Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations

Jun 06, 2019
Debraj Basu, Deepesh Data, Can Karakus, Suhas Diggavi

Figure 1 for Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations

Figure 2 for Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations

Figure 3 for Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computations

* 46 pages, 9 figures

Access Paper or Ask Questions

Densifying Assumed-sparse Tensors: Improving Memory Efficiency and MPI Collective Performance during Tensor Accumulation for Parallelized Training of Neural Machine Translation Models

May 10, 2019
Derya Cavdar, Valeriu Codreanu, Can Karakus, John A. Lockman III, Damian Podareanu, Vikram Saletore, Alexander Sergeev, Don D. Smith II, Victor Suthichai, Quy Ta, Srinivas Varadharajan, Lucas A. Wilson, Rengan Xu, Pei Yang

Figure 1 for Densifying Assumed-sparse Tensors: Improving Memory Efficiency and MPI Collective Performance during Tensor Accumulation for Parallelized Training of Neural Machine Translation Models

Figure 2 for Densifying Assumed-sparse Tensors: Improving Memory Efficiency and MPI Collective Performance during Tensor Accumulation for Parallelized Training of Neural Machine Translation Models

Figure 3 for Densifying Assumed-sparse Tensors: Improving Memory Efficiency and MPI Collective Performance during Tensor Accumulation for Parallelized Training of Neural Machine Translation Models

Figure 4 for Densifying Assumed-sparse Tensors: Improving Memory Efficiency and MPI Collective Performance during Tensor Accumulation for Parallelized Training of Neural Machine Translation Models

* 18 pages, 10 figures, accepted at the 2019 International Supercomputing Conference

Access Paper or Ask Questions

Privacy-Utility Trade-off of Linear Regression under Random Projections and Additive Noise

Feb 13, 2019
Mehrdad Showkatbakhsh, Can Karakus, Suhas Diggavi

Figure 1 for Privacy-Utility Trade-off of Linear Regression under Random Projections and Additive Noise

Figure 2 for Privacy-Utility Trade-off of Linear Regression under Random Projections and Additive Noise

Figure 3 for Privacy-Utility Trade-off of Linear Regression under Random Projections and Additive Noise

* A short version is published in ISIT 2018

Access Paper or Ask Questions

Redundancy Techniques for Straggler Mitigation in Distributed Optimization and Learning

Mar 14, 2018
Can Karakus, Yifan Sun, Suhas Diggavi, Wotao Yin

Figure 1 for Redundancy Techniques for Straggler Mitigation in Distributed Optimization and Learning

Figure 2 for Redundancy Techniques for Straggler Mitigation in Distributed Optimization and Learning

Figure 3 for Redundancy Techniques for Straggler Mitigation in Distributed Optimization and Learning

Figure 4 for Redundancy Techniques for Straggler Mitigation in Distributed Optimization and Learning

* 39 pages, 14 figures. Submitted for publication

Access Paper or Ask Questions

Straggler Mitigation in Distributed Optimization Through Data Encoding

Jan 22, 2018
Can Karakus, Yifan Sun, Suhas Diggavi, Wotao Yin

Figure 1 for Straggler Mitigation in Distributed Optimization Through Data Encoding

Figure 2 for Straggler Mitigation in Distributed Optimization Through Data Encoding

Figure 3 for Straggler Mitigation in Distributed Optimization Through Data Encoding

Figure 4 for Straggler Mitigation in Distributed Optimization Through Data Encoding

* appeared at NIPS 2017

Access Paper or Ask Questions