Shenggui Li

According to our database, Shenggui Li authored at least 14 papers between 2021 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

GliDe with a CaPE: A Low-Hassle Method to Accelerate Speculative Decoding.

Cunxiao Du, Jing Jiang, Yuanchen Xu, Jiawei Wu, Sicheng Yu, Yongqi Li, Shenggui Li, Kai Xu, Liqiang Nie, Zhaopeng Tu, Yang You

Proceedings of the Forty-first International Conference on Machine Learning, 2024

2023

Parallel Training of Pre-Trained Models via Chunk-Based Dynamic Memory Management.

Jiarui Fang, Zilin Zhu, Shenggui Li, Hui Su, Yang Yu, Jie Zhou, Yang You

IEEE Trans. Parallel Distributed Syst., 2023

Colossal-Auto: Unified Automation of Parallelization and Activation Checkpoint for Large-scale Models.

Yuliang Liu, Shenggui Li, Jiarui Fang, Yanjun Shao, Boyuan Yao, Yang You

CoRR, 2023

Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Training.

Shenggui Li, Hongxin Liu, Zhengda Bian, Jiarui Fang, Haichen Huang, Yuliang Liu, Boxiang Wang, Yang You

Proceedings of the 52nd International Conference on Parallel Processing, 2023

Sequence Parallelism: Long Sequence Training from System Perspective.

Shenggui Li, Fuzhao Xue, Chaitanya Baranwal, Yongbin Li, Yang You

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023

2022

Critique of "MemXCT: Memory-Centric X-Ray CT Reconstruction With Massive Parallelization" by SCC Team From Nanyang Technological University.

Shenggui Li, Bu-Sung Lee

IEEE Trans. Parallel Distributed Syst., 2022

Elixir: Train a Large Language Model on a Small GPU Cluster.

Haichen Huang, Jiarui Fang, Hongxin Liu, Shenggui Li, Yang You

CoRR, 2022

EnergonAI: An Inference System for 10-100 Billion Parameter Transformer Models.

Jiangsu Du, Ziming Liu, Jiarui Fang, Shenggui Li, Yongbin Li, Yutong Lu, Yang You

CoRR, 2022

A Frequency-aware Software Cache for Large Recommendation System Embeddings.

Jiarui Fang, Geng Zhang, Jiatong Han, Shenggui Li, Zhengda Bian, Yongbin Li, Jin Liu, Yang You

CoRR, 2022

Sky Computing: Accelerating Geo-distributed Computing in Federated Learning.

Jie Zhu, Shenggui Li, Yang You

CoRR, 2022

2021

PatrickStar: Parallel Training of Pre-trained Models via a Chunk-based Memory Management.

Jiarui Fang, Yang Yu, Shenggui Li, Yang You, Jie Zhou

CoRR, 2021

Sequence Parallelism: Making 4D Parallelism Possible.

Shenggui Li, Fuzhao Xue, Yongbin Li, Yang You

CoRR, 2021

An Efficient 2D Method for Training Super-Large Deep Learning Models.

Qifan Xu, Shenggui Li, Chaoyu Gong, Yang You

CoRR, 2021

Online evolutionary batch size orchestration for scheduling deep learning workloads in GPU clusters.

Zhengda Bian, Shenggui Li, Wei Wang, Yang You

Proceedings of the International Conference for High Performance Computing, 2021