Shuohuan Wang

According to our database, Shuohuan Wang authored at least 38 papers between 2019 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

Mixture of Hidden-Dimensions Transformer.

Yilong Chen, Junyuan Shang, Zhengyu Zhang, Jiawei Sheng, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu, Haifeng Wang

CoRR, 2024

MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions.

Yekun Chai, Haoran Sun, Huang Fang, Shuohuan Wang, Yu Sun, Hua Wu

CoRR, 2024

Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging.

Tingfeng Hui, Zhenyu Zhang, Shuohuan Wang, Yu Sun, Hua Wu, Sen Su

CoRR, 2024

NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time.

Yilong Chen, Guoxia Wang, Junyuan Shang, Shiyao Cui, Zhenyu Zhang, Tingwen Liu, Shuohuan Wang, Yu Sun, Dianhai Yu, Hua Wu

CoRR, 2024

DHA: Learning Decoupled-Head Attention from Transformer Checkpoints via Adaptive Heads Fusion.

Yilong Chen, Linhao Zhang, Junyuan Shang, Zhenyu Zhang, Tingwen Liu, Shuohuan Wang, Yu Sun

CoRR, 2024

HFT: Half Fine-Tuning for Large Language Models.

Tingfeng Hui, Zhenyu Zhang, Shuohuan Wang, Weiran Xu, Yu Sun, Hua Wu

CoRR, 2024

Dual Modalities of Text: Visual and Textual Generative Pre-training.

Yekun Chai, Qingyi Liu, Jingwu Xiao, Shuohuan Wang, Yu Sun, Hua Wu

CoRR, 2024

On Training Data Influence of GPT Models.

Qingyi Liu, Yekun Chai, Shuohuan Wang, Yu Sun, Qiwei Peng, Keze Wang, Hua Wu

CoRR, 2024

Tool-Augmented Reward Modeling.

Lei Li, Yekun Chai, Shuohuan Wang, Yu Sun, Hao Tian, Ningyu Zhang, Hua Wu

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Autoregressive Pre-Training on Pixels and Texts.

Yekun Chai, Qingyi Liu, Jingwu Xiao, Shuohuan Wang, Yu Sun, Hua Wu

Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024

On Training Data Influence of GPT Models.

Yekun Chai, Qingyi Liu, Shuohuan Wang, Yu Sun, Qiwei Peng, Hua Wu

Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024

NACL: A General and Effective KV Cache Eviction Framework for LLM at Inference Time.

Yilong Chen, Guoxia Wang, Junyuan Shang, Shiyao Cui, Zhenyu Zhang, Tingwen Liu, Shuohuan Wang, Yu Sun, Dianhai Yu, Hua Wu

Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024

LEMON: Reviving Stronger and Smaller LMs from Larger LMs with Linear Parameter Fusion.

Yilong Chen, Junyuan Shang, Zhenyu Zhang, Shiyao Cui, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu

Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024

2023

Tool-Augmented Reward Modeling.

Lei Li, Yekun Chai, Shuohuan Wang, Yu Sun, Hao Tian, Ningyu Zhang, Hua Wu

CoRR, 2023

ERNIE-Music: Text-to-Waveform Music Generation with Diffusion Models.

Pengfei Zhu, Chao Pang, Shuohuan Wang, Yekun Chai, Yu Sun, Hao Tian, Hua Wu

CoRR, 2023

Retrieval-Augmented Domain Adaptation of Language Models.

Benfeng Xu, Chunxu Zhao, Wenbin Jiang, Pengfei Zhu, Songtai Dai, Chao Pang, Zhuo Sun, Shuohuan Wang, Yu Sun

Proceedings of the 8th Workshop on Representation Learning for NLP, 2023

ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages.

Yekun Chai, Shuohuan Wang, Chao Pang, Yu Sun, Hao Tian, Hua Wu

Proceedings of the Findings of the Association for Computational Linguistics: ACL 2023, 2023

2022

X-PuDu at SemEval-2022 Task 6: Multilingual Learning for English and Arabic Sarcasm Detection.

Yaqian Han, Yekun Chai, Shuohuan Wang, Yu Sun, Hongyi Huang, Guanghao Chen, Yitong Xu, Yang Yang

CoRR, 2022

ERNIE-UniX2: A Unified Cross-lingual Cross-modal Framework for Understanding and Generation.

Bin Shan, Yaqian Han, Weichong Yin, Shuohuan Wang, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang

CoRR, 2022

ERNIE-SAT: Speech and Text Joint Pretraining for Cross-Lingual Multi-Speaker Text-to-Speech.

Xiaoran Fan, Chao Pang, Tian Yuan, He Bai, Renjie Zheng, Pengfei Zhu, Shuohuan Wang, Junkun Chen, Zeyu Chen, Liang Huang, Yu Sun, Hua Wu

CoRR, 2022

Nebula-I: A General Framework for Collaboratively Training Deep Learning Models on Low-Bandwidth Cloud Clusters.

Yang Xiang, Zhihua Wu, Weibao Gong, Siyu Ding, Xianjie Mo, Yuang Liu, Shuohuan Wang, Peng Liu, Yongshuai Hou, Long Li, Bin Wang, Shaohuai Shi, Yaqian Han, Yue Yu, Ge Li, Yu Sun, Yanjun Ma, Dianhai Yu

CoRR, 2022

X-PuDu at SemEval-2022 Task 7: A Replaced Token Detection Task Pre-trained Model with Pattern-aware Ensembling for Identifying Plausible Clarifications.

Junyuan Shang, Shuohuan Wang, Yu Sun, Yanjun Yu, Yue Zhou, Li Xiang, Guixiu Yang

Proceedings of the 16th International Workshop on Semantic Evaluation, SemEval@NAACL 2022, 2022

X-PuDu at SemEval-2022 Task 6: Multilingual Learning for English and Arabic Sarcasm Detection.

Ya Han, Yekun Chai, Shuohuan Wang, Yu Sun, Hongyi Huang, Guanghao Chen, Yitong Xu, Yang Yang

Proceedings of the 16th International Workshop on Semantic Evaluation, SemEval@NAACL 2022, 2022

Clip-Tuning: Towards Derivative-free Prompt Learning with a Mixture of Rewards.

Yekun Chai, Shuohuan Wang, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang

Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2022, 2022

2021

ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation.

Shuohuan Wang, Yu Sun, Yang Xiang, Zhihua Wu, Siyu Ding, Weibao Gong, Shikun Feng, Junyuan Shang, Yanbin Zhao, Chao Pang, Jiaxiang Liu, Xuyi Chen, Yuxiang Lu, Weixin Liu, Xi Wang, Yangfan Bai, Qiuliang Chen, Li Zhao, Shiyong Li, Peng Sun, Dianhai Yu, Yanjun Ma, Hao Tian, Hua Wu, Tian Wu, Wei Zeng, Ge Li, Wen Gao, Haifeng Wang

CoRR, 2021

ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation.

Yu Sun, Shuohuan Wang, Shikun Feng, Siyu Ding, Chao Pang, Junyuan Shang, Jiaxiang Liu, Xuyi Chen, Yanbin Zhao, Yuxiang Lu, Weixin Liu, Zhihua Wu, Weibao Gong, Jianzhong Liang, Zhizhou Shang, Peng Sun, Wei Liu, Xuan Ouyang, Dianhai Yu, Hao Tian, Hua Wu, Haifeng Wang

CoRR, 2021

abcbpc at SemEval-2021 Task 7: ERNIE-based Multi-task Model for Detecting and Rating Humor and Offense.

Chao Pang, Xiaoran Fan, Weiyue Su, Xuyi Chen, Shuohuan Wang, Jiaxiang Liu, Xuan Ouyang, Shikun Feng, Yu Sun

Proceedings of the 15th International Workshop on Semantic Evaluation, 2021

ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora.

Xuan Ouyang, Shuohuan Wang, Chao Pang, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021

Correcting Chinese Spelling Errors with Phonetic Pre-training.

Ruiqing Zhang, Chao Pang, Chuanqiang Zhang, Shuohuan Wang, Zhongjun He, Yu Sun, Hua Wu, Haifeng Wang

Proceedings of the Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, 2021

ERNIE-Doc: A Retrospective Long-Document Modeling Transformer.

Siyu Ding, Junyuan Shang, Shuohuan Wang, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021

2020

ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora.

Xuan Ouyang, Shuohuan Wang, Chao Pang, Yu Sun, Hao Tian, Hua Wu, Haifeng Wang

CoRR, 2020

kk2018 at SemEval-2020 Task 9: Adversarial Training for Code-Mixing Sentiment Classification.

Jiaxiang Liu, Xuyi Chen, Shikun Feng, Shuohuan Wang, Xuan Ouyang, Yu Sun, Zhengjie Huang, Weiyue Su

CoRR, 2020

Galileo at SemEval-2020 Task 12: Multi-lingual Learning for Offensive Language Identification Using Pre-trained Language Models.

Shuohuan Wang, Jiaxiang Liu, Xuan Ouyang, Yu Sun

Proceedings of the Fourteenth Workshop on Semantic Evaluation, 2020

Kk2018 at SemEval-2020 Task 9: Adversarial Training for Code-Mixing Sentiment Classification.

Jiaxiang Liu, Xuyi Chen, Shikun Feng, Shuohuan Wang, Xuan Ouyang, Yu Sun, Zhengjie Huang, Weiyue Su

Proceedings of the Fourteenth Workshop on Semantic Evaluation, 2020

ERNIE at SemEval-2020 Task 10: Learning Word Emphasis Selection by Pre-trained Language Model.

Zhengjie Huang, Shikun Feng, Weiyue Su, Xuyi Chen, Shuohuan Wang, Jiaxiang Liu, Xuan Ouyang, Yu Sun

Proceedings of the Fourteenth Workshop on Semantic Evaluation, 2020

ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding.

Yu Sun, Shuohuan Wang, Yu-Kun Li, Shikun Feng, Hao Tian, Hua Wu, Haifeng Wang

Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020

2019

ERNIE: Enhanced Representation through Knowledge Integration.

Yu Sun, Shuohuan Wang, Yu-Kun Li, Shikun Feng, Xuyi Chen, Han Zhang, Xin Tian, Danxiang Zhu, Hao Tian, Hua Wu

CoRR, 2019

OleNet at SemEval-2019 Task 9: BERT based Multi-Perspective Models for Suggestion Mining.

Jiaxiang Liu, Shuohuan Wang, Yu Sun

Proceedings of the 13th International Workshop on Semantic Evaluation, 2019