Samyam Rajbhandari

发表

Fast LSTM Inference by Dynamic Decomposition on Cloud Systems

Kurt Keutzer, Yuxiong He, Yang You, 2019, 2019 IEEE International Conference on Data Mining (ICDM).

Locality Optimizations for Regular and Irregular Applications

Samyam Rajbhandari, Samyam Rajbhandari, 2016 .

A Communication-Optimal Framework for Contracting Distributed Tensors

Sriram Krishnamoorthy, P. Sadayappan, Samyam Rajbhandari, 2014, SC14: International Conference for High Performance Computing, Networking, Storage and Analysis.

APMSqueeze: A Communication Efficient Adam-Preconditioned Momentum SGD Algorithm pdf

Ji Liu, Yuxiong He, Xiangru Lian, 2020, ArXiv.

Optimizing the Four-Index Integral Transform Using Data Movement Lower Bounds Analysis

Sriram Krishnamoorthy, P. Sadayappan, Fabrice Rastello, 2017, PPoPP.

Effective Utilization of Tensor Symmetry in Operation Optimization of Tensor Contraction Expressions

P. Sadayappan, Samyam Rajbhandari, Huaijian Zhang, 2012 .

ZeRO: Memory Optimization Towards Training A Trillion Parameter Models pdf

Olatunji Ruwase, Yuxiong He, Samyam Rajbhandari, 2019, SC.

Learning Intrinsic Sparse Structures within Long Short-term Memory pdf

Fang Liu, Yuxiong He, Wei Wen, 2017, ICLR.

AntMan: Sparse Low-Rank Compression to Accelerate RNN inference pdf

Yuxiong He, Samyam Rajbhandari, Harsh Shrivastava, 2019, ArXiv.

On fusing recursive traversals of K-d trees

Robert J. Harrison, Sriram Krishnamoorthy, P. Sadayappan, 2016, CC.

CAST: Contraction Algorithm for Symmetric Tensors

Sriram Krishnamoorthy, P. Sadayappan, Samyam Rajbhandari, 2014, 2014 43rd International Conference on Parallel Processing.

Optimizing CNNs on Multicores for Scalability, Performance and Goodput

Olatunji Ruwase, Yuxiong He, Trishul M. Chilimbi, 2017, ASPLOS.

A Domain-Specific Compiler for a Parallel Multiresolution Adaptive Numerical Simulation Environment

Robert J. Harrison, Sriram Krishnamoorthy, P. Sadayappan, 2016, SC16: International Conference for High Performance Computing, Networking, Storage and Analysis.

Fast LSTM by dynamic decomposition on cloud and distributed systems

Kurt Keutzer, Yuxiong He, Yang You, 2020, Knowledge and Information Systems.

A framework for load balancing of Tensor Contraction expressions via dynamic task partitioning

Sriram Krishnamoorthy, P. Sadayappan, Samyam Rajbhandari, 2013, 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

International Conference on Computational Science, ICCS 2012

P. Sadayappan, Samyam Rajbhandari, Huaijian Zhang, 2012, ICCS.

Accelerating Large Scale Deep Learning Inference through DeepCPU at Microsoft

Olatunji Ruwase, Minjia Zhang, Yuxiong He, 2019, OpML.

DeepCPU: Serving RNN-based Deep Learning Models 10x Faster

Minjia Zhang, Yuxiong He, Samyam Rajbhandari, 2018, USENIX Annual Technical Conference.

ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep learning

Olatunji Ruwase, Samyam Rajbhandari, Jeff Rasley, 2021, SC21: International Conference for High Performance Computing, Networking, Storage and Analysis.

DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale

Ammar Ahmad Awan, Samyam Rajbhandari, Jeff Rasley, 2022 .

1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed

Xiangru Lian, Ammar Ahmad Awan, Hanlin Tang, 2021, ICML.

1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB’s Convergence Speed pdf

Ammar Ahmad Awan, Hanlin Tang, Samyam Rajbhandari, 2021, 2022 IEEE 29th International Conference on High Performance Computing, Data, and Analytics (HiPC).

Scalable and Efficient MoE Training for Multitask Multilingual Models pdf

Alexandre Muzio, Ammar Ahmad Awan, Hany Hassan Awadalla, 2021, ArXiv.

ZeRO: Memory optimizations Toward Training Trillion Parameter Models

Olatunji Ruwase, Samyam Rajbhandari, Jeff Rasley, 2020, SC20: International Conference for High Performance Computing, Networking, Storage and Analysis.