论文信息 - Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model - 字舞流文

Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

Reza Yazdani Aminabadi | Samyam Rajbhandari | Yuxiong He | M. Shoeybi | M. Patwary | P. LeGresley | J. Casper | Bryan Catanzaro | Rewon Child | Michael Houston | Shrimai Prabhumoye | Xia Song | Zhun Liu | Saurabh Tiwary | V. Korthikanti | J. Bernauer | George Zerveas | Shaden Smith | Brandon Norick | Elton Zhang | R. Child