Sharan Narang

发表

Deep Learning Scaling is Predictable, Empirically pdf

Yang Yang, Newsha Ardalani, Md. Mostofa Ali Patwary, 2017, ArXiv.

Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin pdf

Chong Wang, Xiangang Li, Yi Wang, 2015, ICML.

Mixed Precision Training pdf

Hao Wu, David García, Erich Elsen, 2017, ICLR.

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Colin Raffel, Noam Shazeer, Wei Li, 2019, J. Mach. Learn. Res..

DSD: Regularizing Deep Neural Networks with Dense-Sparse-Dense Training Flow pdf

Song Han, William J. Dally, Erich Elsen, 2016, ArXiv.

Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning

Sercan Ö. Arik, Sharan Narang, Jonathan Raiman, 2017, ICLR.

Deep Voice 3: 2000-Speaker Neural Text-to-Speech pdf

Sercan Ömer Arik, John Miller, Wei Ping, 2017, ICLR 2018.

S CALE E FFICIENTLY : I NSIGHTS FROM P RE - TRAINING AND F INE - TUNING T RANSFORMERS

Hyung Won Chung, Dani Yogatama, W. Fedus, 2022 .

Scaling Instruction-Finetuned Language Models

Andrew M. Dai, Hyung Won Chung, J. Dean, 2022, ArXiv.

Scaling Instruction-Finetuned Language Models

Andrew M. Dai, Hyung Won Chung, J. Dean, 2022, ArXiv.

Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?

Hyung Won Chung, Vinh Q. Tran, Dani Yogatama, 2022, EMNLP.

FCM: Forgetful Causal Masking Makes Causal Language Models Better Zero-Shot Learners

S. Levine, P. Abbeel, Igor Mordatch, 2022, ArXiv.

Understanding HTML with Large Language Models

Ofir Nachum, Sharan Narang, Yingjie Miao, 2022, ArXiv.

Scaling Instruction-Finetuned Language Models

Andrew M. Dai, Hyung Won Chung, J. Dean, 2022, ArXiv.

Character-Aware Models Improve Visual Text Rendering

Daniel H Garrette, Mohammad Norouzi, Noah Constant, 2022, ArXiv.

PaLM: Scaling Language Modeling with Pathways

Andrew M. Dai, Hyung Won Chung, Thanumalayan Sankaranarayana Pillai, 2022, J. Mach. Learn. Res..

Towards Better Few-Shot and Finetuning Performance with Forgetful Causal Language Models

S. Levine, P. Abbeel, Igor Mordatch, 2022, 2210.13432.

Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers

Hyung Won Chung, Dani Yogatama, W. Fedus, 2021, ArXiv.

WT5?! Training Text-to-Text Models to Explain their Predictions pdf

Colin Raffel, Sharan Narang, Adam Roberts, 2020, ArXiv.

Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning

Sercan Ömer Arik, John Miller, Wei Ping, 2017, ICLR.

DSD: Dense-Sparse-Dense Training for Deep Neural Networks

Song Han, William J. Dally, Erich Elsen, 2016, ICLR.

Do Transformer Modifications Transfer Across Implementations and Applications?

Hyung Won Chung, Colin Raffel, Noam M. Shazeer, 2021, EMNLP.

Scaling Up Models and Data with t5x and seqio

Marc van Zee, Hyung Won Chung, Daniel H Garrette, 2022, ArXiv.

Block-Sparse Recurrent Neural Networks pdf

Gregory Frederick Diamos, Sharan Narang, Eric Undersander, 2017, ArXiv.

Exploring Sparsity in Recurrent Neural Networks pdf

Erich Elsen, Greg Diamos, Sharan Narang, 2017, ICLR.

ByT5: Towards a Token-Free Future with Pre-trained Byte-to-Byte Models

Rami Al-Rfou, Colin Raffel, Noah Constant, 2021, Transactions of the Association for Computational Linguistics.

Understanding HTML with Large Language Models

Ofir Nachum, Sharan Narang, Yingjie Miao, 2022, ArXiv.

Character-Aware Models Improve Visual Text Rendering

Daniel H Garrette, Mohammad Norouzi, Noah Constant, 2022, ACL.

Llama 2: Open Foundation and Fine-Tuned Chat Models

Eric Michael Smith, Cristian Canton Ferrer, Daniel M. Bikel, 2023, ArXiv.

Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin

A. Ng, Dario Amodei, P. LeGresley, 2015, ICML.

A Theory on Adam Instability in Large-Scale Machine Learning

Punit Singh Koura, Naman Goyal, Zach DeVito, 2023, ArXiv.

UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining

Hyung Won Chung, Orhan Firat, Noah Constant, 2023, ICLR.

A Proposed Hierarchy of Deep Learning Tasks

Kenneth Ward Church, Yang Yang, Gregory Diamos, 2018 .

MULTITuDE: Large-Scale Multilingual Machine-Generated Text Detection Benchmark

Ona de Gibert, David Ifeoluwa Adelani, M Saiful Bari, 2023, ArXiv.

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Peter J. Liu, Colin Raffel, Noam M. Shazeer, 2019, J. Mach. Learn. Res..

Mixed Precision Training

Boris Ginsburg, Sharan Narang, G. Diamos, 2017, ICLR.

Scaling Instruction-Finetuned Language Models

Andrew M. Dai, Adams Wei Yu, Hyung Won Chung, 2022, ArXiv.

One-Step Diffusion Distillation via Deep Equilibrium Models

Siddhant M. Jayakumar, J. Z. Kolter, Raphael Lopez Kaufman, 2023, NeurIPS.

T ailor : Generating and Perturbing Text with Semantic Controls

Ankur P. Parikh, Amrith Rajagopal Setlur, Yinhan Liu, 2021 .

FarFetched: An Entity-centric Approach for Reasoning on Textually Represented Environments

Noam M. Shazeer, Nikita Nangia, Sharan Narang, 2021 .

When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards

Mitesh M. Khapra, Isabel M. Kloumann, Niladri S. Chatterji, 2024, ArXiv.

Exploring Sparsity in Recurrent Neural Networks

Sharan Narang, G. Diamos, Erich Elsen, 2017, ICLR.