Reza Yazdani Aminabadi

发表

Fault-tolerant 3-D network-on-chip design using dynamic link sharing

Masoud Daneshtalab, Mehdi Modarressi, Seyyed Hossein Seyyedaghaei Rezaei, 2016, 2016 Design, Automation & Test in Europe Conference & Exhibition (DATE).

ZeRO-Offload: Democratizing Billion-Scale Model Training pdf

Olatunji Ruwase, Minjia Zhang, Yuxiong He, 2021, USENIX ATC.

Ultra low-power, high-performance accelerator for speech recognition

Reza Yazdani Aminabadi, 2019 .

DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale

Ammar Ahmad Awan, Samyam Rajbhandari, Jeff Rasley, 2022 .

DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale

Reza Yazdani Aminabadi, Samyam Rajbhandari, Jeff Rasley, 2022, SC22: International Conference for High Performance Computing, Networking, Storage and Analysis.

ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

Reza Yazdani Aminabadi, Yuxiong He, Minjia Zhang, 2022, NeurIPS.

DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale

Reza Yazdani Aminabadi, Samyam Rajbhandari, Jeff Rasley, 2022, ICML.

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Reza Yazdani Aminabadi, Samyam Rajbhandari, Yuxiong He, 2022, ArXiv.

Understanding INT4 Quantization for Transformer Models: Latency Speedup, Composability, and Failure Cases

Reza Yazdani Aminabadi, Yuxiong He, Z. Yao, 2023, ArXiv.

Understanding Int4 Quantization for Language Models: Latency Speedup, Composability, and Failure Cases

Reza Yazdani Aminabadi, Yuxiong He, Z. Yao, 2023, ICML.

DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales

Reza Yazdani Aminabadi, Samyam Rajbhandari, Jeff Rasley, 2023, ArXiv.

ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks

Reza Yazdani Aminabadi, Olatunji Ruwase, Xiaoxia Wu, 2023, 2312.08583.

ZeroQuant-HERO: Hardware-Enhanced Robust Optimized Post-Training Quantization Framework for W8A8 Transformers

Reza Yazdani Aminabadi, Elton Zheng, Xiaoxia Wu, 2023, ArXiv.

DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference

Reza Yazdani Aminabadi, Samyam Rajbhandari, Jeff Rasley, 2024, ArXiv.