Dongsheng Li

发表

Merak: An Efficient Distributed DNN Training Framework with Automated 3D Parallelism for Giant Foundation Models

Linbo Qiao, Xudong Tang, Zhiquan Lai, 2022, ArXiv.

EmbRace: Accelerating Sparse Communication for Distributed Training of Deep Neural Networks

Zhiquan Lai, Shengwei Li, Yabo Duan, 2021, ICPP.

AutoPipe: A Fast Pipeline Parallelism Approach with Balanced Partitioning and Micro-batch Slicing

Zhiquan Lai, Shengwei Li, Ke-shi Ge, 2022, IEEE International Conference on Cluster Computing.

DELTA: Dynamically Optimizing GPU Memory beyond Tensor Recomputation

Xingcheng Zhang, Yuliang Liu, Linbo Qiao, 2022, ArXiv.

Compressed Collective Sparse-Sketch for Distributed Data-Parallel Training of Deep Learning Models

Zhiquan Lai, Ke-shi Ge, Dongsheng Li, 2023, IEEE Journal on Selected Areas in Communications.

Scalable and Explainable 1-Bit Matrix Completion via Graph Signal Learning

Junchi Yan, Xiaokang Yang, Dongsheng Li, 2021, AAAI.

BRGraph: An efficient graph neural network training system by reusing batch data on GPU

Zhiquan Lai, Ke-shi Ge, Dongsheng Li, 2022, Concurr. Comput. Pract. Exp..

PCGraph: Accelerating GNN Inference on Large Graphs via Partition Caching

Zhiquan Lai, Feng Liu, Yu Tang, 2021, 2021 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom).

Automated Tensor Model Parallelism with Overlapped Communication for Efficient Foundation Model Training

Zhiquan Lai, Shengwei Li, Ke-shi Ge, 2023, ArXiv.

Prediction of the Cyanobacteria Coverage in Time-series Images based on Convolutional Neural Network

Zhiquan Lai, Dongsheng Li, Xiangyu Ye, 2021, ICCCV.

Hippie: A Data-Paralleled Pipeline Approach to Improve Memory-Efficiency and Scalability for Large DNN Training

Zhiquan Lai, Linbo Qiao, Dongsheng Li, 2021, ICPP.

A Survey on Auto-Parallelism of Large-Scale Deep Learning Training

Teng Su, Xiaoda Zhang, Zhiquan Lai, 2023, IEEE Transactions on Parallel and Distributed Systems.

Hierarchical Adaptive Pooling by Capturing High-Order Dependency for Graph Representation Learning pdf

Ning Liu, Zhiquan Lai, Songlei Jian, 2021, IEEE Transactions on Knowledge and Data Engineering.

2PGraph: Accelerating GNN Training over Large Graphs on GPU Clusters

Zhiquan Lai, Feng Liu, Yu Tang, 2021, 2021 IEEE International Conference on Cluster Computing (CLUSTER).

NENN: Incorporate Node and Edge Features in Graph Neural Networks

Dongsheng Li, Yulei Yang, Dongsheng Li, 2020, ACML.

SCGraph: Accelerating Sample-based GNN Training by Staged Caching of Features on GPUs

Zhiquan Lai, Dongsheng Li, Zhejiang Ran, 2022, 2022 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom).

MLPs: Efficient Training of MiniGo on Large-scale Heterogeneous Computing System

Y. Dou, Rongchun Li, Dongsheng Li, 2023, 2022 IEEE 28th International Conference on Parallel and Distributed Systems (ICPADS).

P-ADMMiRNN: Training RNN with Stable Convergence via An Efficient and Paralleled ADMM Approach

Linbo Qiao, Zhiquan Lai, Dongsheng Li, 2020, 2006.05622.

Merak: An Efficient Distributed DNN Training Framework With Automated 3D Parallelism for Giant Foundation Models

Linbo Qiao, Xudong Tang, Zhiquan Lai, 2022, IEEE Transactions on Parallel and Distributed Systems.

HPH: Hybrid Parallelism on Heterogeneous Clusters for Accelerating Large-scale DNNs Training

Dongsheng Li, Zhiquan Lai, Shengwei Li, 2022, 2022 IEEE International Conference on Cluster Computing (CLUSTER).

A Memory-Efficient Hybrid Parallel Framework for Deep Neural Network Training

Zhiquan Lai, Shengwei Li, Dongsheng Li, 2024, IEEE Transactions on Parallel and Distributed Systems.

Prophet: Fine-grained Load Balancing for Parallel Training of Large-scale MoE Models

Zhiquan Lai, Shengwei Li, Ke-shi Ge, 2023, 2023 IEEE International Conference on Cluster Computing (CLUSTER).

Deep-to-Bottom Weights Decay: A Systemic Knowledge Review Learning Technique for Transformer Layers in Knowledge Distillation

Minghao Hu, Feng Liu, Yifan Chen, 2022, KSEM.

Accurate and Efficient Fine-Tuning of Quantized Large Language Models Through Optimal Balance

Zhiquan Lai, Dongsheng Li, Ao Shen, 2024, ArXiv.

A Multidimensional Communication Scheduling Method for Hybrid Parallel DNN Training

Zhiquan Lai, Ke-shi Ge, Dongsheng Li, 2024, IEEE Transactions on Parallel and Distributed Systems.