Minjia Zhang

发表

Learning Intrinsic Sparse Structures within Long Short-term Memory pdf

Fang Liu, Yuxiong He, Wei Wen, 2017, ICLR.

Memcached Design on High Performance RDMA Capable Interconnects

Sayantan Sur, Dhabaleswar K. Panda, Xiangyong Ouyang, 2011, 2011 International Conference on Parallel Processing.

Bamboo: Making Preemptible Instances Resilient for Affordable Training of Large DNNs

Zhihao Jia, Minjia Zhang, R. Netravali, 2022, NSDI.

All That Glitters Is Not Gold: Improving Availability and Practicality of Exception-Based Memory Models

Minjia Zhang, 2016 .

Drinking from Both Glasses : Adaptively Combining Pessimistic and Optimistic Synchronization for Efficient Parallel Runtime Support ∗

Michael D. Bond, Minjia Zhang, Man Cao, 2014 .

DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale

Reza Yazdani Aminabadi, Samyam Rajbhandari, Jeff Rasley, 2022, SC22: International Conference for High Performance Computing, Networking, Storage and Analysis.

ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

Reza Yazdani Aminabadi, Yuxiong He, Minjia Zhang, 2022, NeurIPS.

Extreme Compression for Pre-trained Transformers Made Simple and Efficient

Yuxiong He, Minjia Zhang, Conglong Li, 2022, NeurIPS.

Compressing Pre-trained Transformers via Low-Bit NxM Sparsity for Natural Language Understanding

Yuxiong He, Minjia Zhang, Connor Holmes, 2022, ArXiv.

Adversarial Data Augmentation for Task-Specific Knowledge Distillation of Pre-trained Transformers

Yuxiong He, Minjia Zhang, Niranjan Uma Naresh, 2022, AAAI.

Sentinel: Efficient Tensor Migration and Allocation on Heterogeneous Memory Systems for Deep Learning

Minjia Zhang, Kai Wu, Hyeran Jeon, 2021, 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA).

The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models

Yuxiong He, Minjia Zhang, Conglong Li, 2021, NeurIPS.

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Alexander M. Rush, Dragomir R. Radev, Ona de Gibert, 2022, ArXiv.

Random-LTD: Random and Layerwise Token Dropping Brings Efficient Training for Large-scale Transformers

Yuxiong He, Minjia Zhang, Conglong Li, 2022, ArXiv.

DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing

Yuxiong He, Minjia Zhang, Conglong Li, 2022, 2212.03597.

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Alexander M. Rush, Dragomir R. Radev, Ona de Gibert, 2022, ArXiv.

SHARP: An Adaptable, Energy-Efficient Accelerator for Recurrent Neural Networks

Olatunji Ruwase, Yuxiong He, Minjia Zhang, 2019, ACM Transactions on Embedded Computing Systems.

LSTM-Sharp: An Adaptable, Energy-Efficient Hardware Accelerator for Long Short-Term Memory pdf

Reza Yazdani, Olatunji Ruwase, Minjia Zhang, 2019, ArXiv.

ZeRO-Offload: Democratizing Billion-Scale Model Training pdf

Olatunji Ruwase, Minjia Zhang, Yuxiong He, 2021, USENIX ATC.

DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing

Yuxiong He, Minjia Zhang, Conglong Li, 2022, AAAI.

Navigating with Graph Representations for Fast and Scalable Decoding of Neural Language Models

Jianfeng Gao, Minjia Zhang, Yuxiong He, 2018, NeurIPS.

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Alexander M. Rush, Dragomir R. Radev, Ona de Gibert, 2022, ArXiv.

GRIP: Multi-Store Capacity-Optimized High-Performance Nearest Neighbor Search for Vector Search Engine

Minjia Zhang, Yuxiong He, Yuxiong He, 2019, CIKM.

Betty: Enabling Large-Scale GNN Training with Batch-Level Graph Partitioning

Minjia Zhang, Wenqian Dong, Dong Li, 2023, ASPLOS.

Accelerating Training of Transformer-Based Language Models with Progressive Layer Dropping pdf

Minjia Zhang, Yuxiong He, Yuxiong He, 2020, NeurIPS.

HM-ANN: Efficient Billion-Point Nearest Neighbor Search on Heterogeneous Memory

Minjia Zhang, Jie Ren, Dong Li, 2020, NeurIPS.

DeepCPU: Serving RNN-based Deep Learning Models 10x Faster

Minjia Zhang, Yuxiong He, Samyam Rajbhandari, 2018, USENIX Annual Technical Conference.

Estimating Available Bandwidth Using Multiple Overloading Streams

Jiang Li, Chong Luo, Minjian Zhang, 2006, 2006 IEEE International Conference on Communications.

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Alexander M. Rush, Dragomir R. Radev, Ona de Gibert, 2022, ArXiv.

Avoiding consistency exceptions under strong memory models

Swarnendu Biswas, Minjia Zhang, Michael D. Bond, 2017, ISMM.

Lightweight data race detection for production runs

Benjamin P. Wood, Swarnendu Biswas, Minjia Zhang, 2017, CC.

Valor: efficient, software-only region conflict exceptions

Swarnendu Biswas, Minjia Zhang, Michael D. Bond, 2015, OOPSLA.

Hybrid Static–Dynamic Analysis for Statically Bounded Region Serializability

Swarnendu Biswas, Minjia Zhang, Michael D. Bond, 2015, ASPLOS.

OCTET: capturing and controlling cross-thread dependences efficiently

Swarnendu Biswas, Minjia Zhang, Michael D. Bond, 2013, OOPSLA.

Drinking from Both Glasses : Adaptively Combining Pessimistic and Optimistic Synchronization for Efficient Parallel Runtime Support ∗

Michael D. Bond, Minjia Zhang, Man Cao, 2014 .

SHARP: An Adaptable, Energy-Efficient Accelerator for Recurrent Neural Networks

Olatunji Ruwase, Yuxiong He, Minjia Zhang, 2019, ACM Trans. Embed. Comput. Syst..

DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales

Reza Yazdani Aminabadi, Samyam Rajbhandari, Jeff Rasley, 2023, ArXiv.

Improving Approximate Nearest Neighbor Search through Learned Adaptive Early Termination

Minjia Zhang, Yuxiong He, David G. Andersen, 2020, SIGMOD Conference.

iQAN: Fast and Accurate Vector Search with Efficient Intra-Query Parallelism on Multi-Core Architectures

K. Li, Minjia Zhang, R. Jin, 2023, PPoPP.

iQAN

K. Li, Minjia Zhang, R. Jin, 2023, Proceedings of the 28th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming.

AdaTune: Adaptive Tensor Program Compilation Made Efficient

Minjia Zhang, Chi Wang, Mingqin Li, 2020, NeurIPS.

Stack-Aware Hyperproperties

Mahesh Viswanathan, Minjia Zhang, Rohit Chadha, 2023, TACAS.

DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention

Samyam Rajbhandari, Olatunji Ruwase, Yuxiong He, 2023, ArXiv.

DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models

S. A. Jacobs, Samyam Rajbhandari, Yuxiong He, 2023, ArXiv.

Accelerating Large Scale Deep Learning Inference through DeepCPU at Microsoft

Olatunji Ruwase, Minjia Zhang, Yuxiong He, 2019, OpML.

Cost-effective On-device Continual Learning over Memory Hierarchy with Miro

Minjia Zhang, Myeongjae Jeon, Jonghyun Choi, 2023, MobiCom.

Dynamic Race Detection with O(1) Samples

Mahesh Viswanathan, Minjia Zhang, Umang Mathur, 2023, Proc. ACM Program. Lang..

SIRe: an efficient snapshot isolation-based memory model for detecting and tolerating region conflicts

Minjia Zhang, Minjia Zhang, 2015, SPLASH.