Noam Shazeer

发表

An Improved Relative Self-Attention Mechanism for Transformer with Application to Music Generation pdf

Douglas Eck, Andrew M. Dai, Cheng-Zhi Anna Huang, 2018, ArXiv.

GLU Variants Improve Transformer pdf

Noam Shazeer, Noam M. Shazeer, 2020, ArXiv.

Image Transformer pdf

Dustin Tran, Lukasz Kaiser, Ashish Vaswani, 2018, ICML.

Swivel: Improving Embeddings by Noticing What's Missing pdf

Noam Shazeer, Ryan Doherty, Colin Evans, 2016, ArXiv.

Sparse non-negative matrix language modeling for geo-annotated query session data

Noam Shazeer, Ciprian Chelba, 2015, 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU).

End-to-end text-dependent speaker verification

Georg Heigold, Samy Bengio, Noam Shazeer, 2015, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

Large Scale Multi-Domain Multi-Task Learning with MultiModel

Lukasz Kaiser, Ashish Vaswani, Jakob Uszkoreit, 2018 .

Talking-Heads Attention pdf

Noam Shazeer, Le Hou, Nan Ding, 2020, ArXiv.

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding pdf

Orhan Firat, Zhifeng Chen, Noam Shazeer, 2020, ICLR.

One Model To Learn Them All pdf

Lukasz Kaiser, Ashish Vaswani, Jakob Uszkoreit, 2017, ArXiv.

Pruning sparse non-negative matrix n-gram language models

Joris Pelemans, Noam Shazeer, Ciprian Chelba, 2015, INTERSPEECH.

Fast Transformer Decoding: One Write-Head is All You Need pdf

Noam Shazeer, Noam M. Shazeer, 2019, ArXiv.

High Resolution Medical Image Analysis with Spatial Partitioning pdf

Travis M. Drucker, Noam Shazeer, Panagiotis Korfiatis, 2019, ArXiv.

Mesh-TensorFlow: Deep Learning for Supercomputers

Dustin Tran, Ashish Vaswani, Noam Shazeer, 2018, NeurIPS.

Tensor2Tensor for Neural Machine Translation

Samy Bengio, Lukasz Kaiser, Ashish Vaswani, 2018, AMTA.

Generating Wikipedia by Summarizing Long Sequences pdf

Lukasz Kaiser, Noam Shazeer, Ben Goodrich, 2018, ICLR.

How Much Knowledge Can You Pack Into the Parameters of a Language Model? pdf

Colin Raffel, Noam Shazeer, Adam Roberts, 2020, EMNLP.

Sparse Non-negative Matrix Language Modeling (EMNLP presentation)

Joris Pelemans, Noam Shazeer, Ciprian Chelba, 2016 .

HydraNets: Specialized Dynamic Architectures for Efficient Inference

Noam Shazeer, William R. Mark, Kayvon Fatahalian, 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

Weakly Supervised Grammatical Error Correction using Iterative Decoding pdf

Shankar Kumar, Noam Shazeer, Niki Parmar, 2018, ArXiv.

Sparse Non-negative Matrix Language Modeling

Joris Pelemans, Noam Shazeer, Ciprian Chelba, 2016, Transactions of the Association for Computational Linguistics.

Exploring the Limits of Language Modeling pdf

Yonghui Wu, Oriol Vinyals, Noam Shazeer, 2016, ArXiv.

Blockwise Parallel Decoding for Deep Autoregressive Models

Jakob Uszkoreit, Noam Shazeer, Mitchell Stern, 2018, NeurIPS.

Attention is All you Need pdf

Lukasz Kaiser, Ashish Vaswani, Jakob Uszkoreit, 2017, NIPS.

Music Transformer: Generating Music with Long-Term Structure

Andrew M. Dai, Cheng-Zhi Anna Huang, Monica Dinculescu, 2018, ICLR.

Music Transformer

Douglas Eck, Andrew M. Dai, Cheng-Zhi Anna Huang, 2018, 1809.04281.

Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer pdf

Geoffrey E. Hinton, Quoc V. Le, Azalia Mirhoseini, 2017, ICLR.

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Colin Raffel, Noam Shazeer, Wei Li, 2019, J. Mach. Learn. Res..

Fast Decoding in Sequence Models using Discrete Latent Variables

Aurko Roy, Samy Bengio, Ashish Vaswani, 2018, ICML.

Faster Transformer Decoding: N-gram Masked Self-Attention pdf

Ankur Bapna, Noam Shazeer, Ciprian Chelba, 2020, ArXiv.

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity pdf

Noam Shazeer, Barret Zoph, William Fedus, 2021, ArXiv.

Adafactor: Adaptive Learning Rates with Sublinear Memory Cost pdf

Noam Shazeer, Mitchell Stern, Noam M. Shazeer, 2018, ICML.

Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks

Samy Bengio, Navdeep Jaitly, Oriol Vinyals, 2015, NIPS.

NN-Grams: Unifying Neural Network and n-Gram Language Models for Speech Recognition

Shankar Kumar, Noam Shazeer, Antoine Bruguier, 2016, INTERSPEECH.

Sparse non-negative matrix language modeling for skip-grams

Joris Pelemans, Noam Shazeer, Ciprian Chelba, 2015, INTERSPEECH.

Skip-gram Language Modeling Using Sparse Non-negative Matrix Probability Estimation pdf

Joris Pelemans, Noam Shazeer, Ciprian Chelba, 2014, ArXiv.

Primer: Searching for Efficient Transformers for Language Modeling pdf

Quoc V. Le, Hanxiao Liu, Noam Shazeer, 2021, NeurIPS.

GSPMD: General and Scalable Parallelization for ML Computation Graphs pdf

Noam Shazeer, Dehao Chen, Yonghui Wu, 2021, ArXiv.

LaMDA: Language Models for Dialog Applications

Lora Aroyo, Heng-Tze Cheng, Ben Hutchinson, 2022 .

Do Transformer Modifications Transfer Across Implementations and Applications? pdf

Colin Raffel, Wei Li, Noam Shazeer, 2021, EMNLP.