Hongyu Wang

According to our database, Hongyu Wang authored at least 8 papers between 2022 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

DeepNet: Scaling Transformers to 1,000 Layers.

Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Furu Wei

IEEE Trans. Pattern Anal. Mach. Intell., October, 2024

Q-Sparse: All Large Language Models can be Fully Sparsely-Activated.

Hongyu Wang, Shuming Ma, Ruiping Wang, Furu Wei

CoRR, 2024

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits.

Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, Furu Wei

CoRR, 2024

2023

BitNet: Scaling 1-bit Transformers for Large Language Models.

Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, Furu Wei

CoRR, 2023

Magneto: A Foundation Transformer.

Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, Furu Wei

Proceedings of the International Conference on Machine Learning, 2023

2022

TorchScale: Transformers at Scale.

Shuming Ma, Hongyu Wang, Shaohan Huang, Wenhui Wang, Zewen Chi, Li Dong, Alon Benhaim, Barun Patra, Vishrav Chaudhary, Xia Song, Furu Wei

CoRR, 2022

Foundation Transformers.

Hongyu Wang, Shuming Ma, Shaohan Huang, Li Dong, Wenhui Wang, Zhiliang Peng, Yu Wu, Payal Bajaj, Saksham Singhal, Alon Benhaim, Barun Patra, Zhun Liu, Vishrav Chaudhary, Xia Song, Furu Wei

CoRR, 2022

DeepNet: Scaling Transformers to 1, 000 Layers.

Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Dongdong Zhang, Furu Wei

CoRR, 2022