Li Dong

Orcid: 0000-0003-3083-7170

Affiliations:

Microsoft Research Asia, Natural Language Computing Group, Beijing, China
University of Edinburgh, School of Informatics, Edinburgh, UK (PhD 2019)
Beihang University, State Key Laboratory of Software Development Environment, Beijing, China (former)

According to our database¹, Li Dong authored at least 124 papers between 2011 and 2024.

Collaborative distances:

Dijkstra number² of four.
Erdős number³ of three.

Timeline

Legend:

Book

In proceedings

Article

PhD thesis

Dataset

Other

Bibliography

2024

DeepNet: Scaling Transformers to 1,000 Layers.

[BibT_eX]

[DOI]

IEEE Trans. Pattern Anal. Mach. Intell., October, 2024

Generic-to-Specific Distillation of Masked Autoencoders.

[BibT_eX]

[DOI]

IEEE Trans. Circuits Syst. Video Technol., September, 2024

Fine-tuning pretrained transformer encoders for sequence-to-sequence learning.

[BibT_eX]

[DOI]

Int. J. Mach. Learn. Cybern., May, 2024

RedStone: Curating General, Code, Math, and QA Data for Large Language Models.

[BibT_eX]

[DOI]

CoRR, 2024

Data Selection via Optimal Control for Language Models.

[BibT_eX]

[DOI]

CoRR, 2024

Self-Boosting Large Language Models with Synthetic Preference Data.

[BibT_eX]

[DOI]

CoRR, 2024

Differential Transformer.

[BibT_eX]

[DOI]

CoRR, 2024

You Only Cache Once: Decoder-Decoder Architectures for Language Models.

[BibT_eX]

[DOI]

CoRR, 2024

Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models.

[BibT_eX]

[DOI]

CoRR, 2024

The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits.

[BibT_eX]

[DOI]

CoRR, 2024

Towards Optimal Learning of Language Models.

[BibT_eX]

[DOI]

CoRR, 2024

Grounding Multimodal Large Language Models to the World.

[BibT_eX]

[DOI]

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Kosmos-G: Generating Images in Context with Multimodal Large Language Models.

[BibT_eX]

[DOI]

Proceedings of the Twelfth International Conference on Learning Representations, 2024

MiniLLM: Knowledge Distillation of Large Language Models.

[BibT_eX]

[DOI]

Proceedings of the Twelfth International Conference on Learning Representations, 2024

Language Models as Inductive Reasoners.

[BibT_eX]

[DOI]

Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, 2024

2023

A Unified View of Masked Image Modeling.

[BibT_eX]

[DOI]

Trans. Mach. Learn. Res., 2023

BitNet: Scaling 1-bit Transformers for Large Language Models.

[BibT_eX]

[DOI]

CoRR, 2023

Kosmos-2.5: A Multimodal Literate Model.

[BibT_eX]

[DOI]

CoRR, 2023

Large Language Model for Science: A Study on P vs. NP.

[BibT_eX]

[DOI]

CoRR, 2023

Retentive Network: A Successor to Transformer for Large Language Models.

[BibT_eX]

[DOI]

CoRR, 2023

LongNet: Scaling Transformers to 1, 000, 000, 000 Tokens.

[BibT_eX]

[DOI]

CoRR, 2023

Kosmos-2: Grounding Multimodal Large Language Models to the World.

[BibT_eX]

[DOI]

CoRR, 2023

Knowledge Distillation of Large Language Models.

[BibT_eX]

[DOI]

CoRR, 2023

Augmenting Language Models with Long-Term Memory.

[BibT_eX]

[DOI]

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Language Is Not All You Need: Aligning Perception with Language Models.

[BibT_eX]

[DOI]

Nils Johan Bertil Bjorck

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Optimizing Prompts for Text-to-Image Generation.

[BibT_eX]

[DOI]

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Extensible Prompts for Language Models on Zero-shot Language Style Customization.

[BibT_eX]

[DOI]

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Magneto: A Foundation Transformer.

[BibT_eX]

[DOI]

Proceedings of the International Conference on Machine Learning, 2023

Visually-Augmented Language Modeling.

[BibT_eX]

[DOI]

Proceedings of the Eleventh International Conference on Learning Representations, 2023

Prototypical Calibration for Few-shot Learning of Language Models.

[BibT_eX]

[DOI]

Proceedings of the Eleventh International Conference on Learning Representations, 2023

Corrupted Image Modeling for Self-Supervised Visual Pre-Training.

[BibT_eX]

[DOI]

Proceedings of the Eleventh International Conference on Learning Representations, 2023

Non-Contrastive Learning Meets Language-Image Pre-Training.

[BibT_eX]

[DOI]

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

Image as a Foreign Language: BEIT Pretraining for Vision and Vision-Language Tasks.

[BibT_eX]

[DOI]

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator.

[BibT_eX]

[DOI]

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023

A Length-Extrapolatable Transformer.

[BibT_eX]

[DOI]

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023

Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning.

[BibT_eX]

[DOI]

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023

Pre-Training to Learn in Context.

[BibT_eX]

[DOI]

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023

Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers.

[BibT_eX]

[DOI]

Proceedings of the Findings of the Association for Computational Linguistics: ACL 2023, 2023

2022

Transforming Wikipedia Into Augmented Data for Query-Focused Summarization.

[BibT_eX]

[DOI]

IEEE ACM Trans. Audio Speech Lang. Process., 2022

Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers.

[BibT_eX]

[DOI]

CoRR, 2022

Structured Prompting: Scaling In-Context Learning to 1, 000 Examples.

[BibT_eX]

[DOI]

CoRR, 2022

Extensible Prompts for Language Models.

[BibT_eX]

[DOI]

CoRR, 2022

TorchScale: Transformers at Scale.

[BibT_eX]

[DOI]

CoRR, 2022

Foundation Transformers.

[BibT_eX]

[DOI]

CoRR, 2022

Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks.

[BibT_eX]

[DOI]

CoRR, 2022

BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers.

[BibT_eX]

[DOI]

CoRR, 2022

Language Models are General-Purpose Interfaces.

[BibT_eX]

[DOI]

CoRR, 2022

VL-BEiT: Generative Vision-Language Pretraining.

[BibT_eX]

[DOI]

CoRR, 2022

Prototypical Calibration for Few-shot Learning of Language Models.

[BibT_eX]

[DOI]

CoRR, 2022

On the Representation Collapse of Sparse Mixture of Experts.

[BibT_eX]

[DOI]

CoRR, 2022

DeepNet: Scaling Transformers to 1, 000 Layers.

[BibT_eX]

[DOI]

CoRR, 2022

A Survey of Knowledge-Intensive NLP with Pre-Trained Language Models.

[BibT_eX]

[DOI]

CoRR, 2022

Kformer: Knowledge Injection in Transformer Feed-Forward Layers.

[BibT_eX]

[DOI]

Proceedings of the Natural Language Processing and Chinese Computing, 2022

On the Representation Collapse of Sparse Mixture of Experts.

[BibT_eX]

[DOI]

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts.

[BibT_eX]

[DOI]

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

BEiT: BERT Pre-Training of Image Transformers.

[BibT_eX]

[DOI]

Proceedings of the Tenth International Conference on Learning Representations, 2022

CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation.

[BibT_eX]

[DOI]

Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2022, 2022

Swin Transformer V2: Scaling Up Capacity and Resolution.

[BibT_eX]

[DOI]

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

Controllable Natural Language Generation with Contrastive Prefixes.

[BibT_eX]

[DOI]

Proceedings of the Findings of the Association for Computational Linguistics: ACL 2022, 2022

Knowledge Neurons in Pretrained Transformers.

[BibT_eX]

[DOI]

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022

StableMoE: Stable Routing Strategy for Mixture of Experts.

[BibT_eX]

[DOI]

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022

XLM-E: Cross-lingual Language Model Pre-training via ELECTRA.

[BibT_eX]

[DOI]

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022

THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption.

[BibT_eX]

[DOI]

Proceedings of the Findings of the Association for Computational Linguistics: ACL 2022, 2022

CLIP Models are Few-Shot Learners: Empirical Studies on VQA and Visual Entailment.

[BibT_eX]

[DOI]

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022

2021

VLMo: Unified Vision-Language Pre-Training with Mixture-of-Modality-Experts.

[BibT_eX]

[DOI]

CoRR, 2021

s2s-ft: Fine-Tuning Pretrained Transformer Encoders for Sequence-to-Sequence Learning.

[BibT_eX]

[DOI]

CoRR, 2021

XLM-E: Cross-lingual Language Model Pre-training via ELECTRA.

[BibT_eX]

[DOI]

CoRR, 2021

DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders.

[BibT_eX]

[DOI]

CoRR, 2021

BEiT: BERT Pre-Training of Image Transformers.

[BibT_eX]

[DOI]

Hangbo Bao

Li Dong

Furu Wei

CoRR, 2021

Knowledge Neurons in Pretrained Transformers.

[BibT_eX]

[DOI]

CoRR, 2021

mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs.

[BibT_eX]

[DOI]

CoRR, 2021

Learning natural language interfaces with neural models.

[BibT_eX]

[DOI]

Li Dong

AI Matters, 2021

Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task.

[BibT_eX]

[DOI]

Proceedings of the Sixth Conference on Machine Translation, 2021

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training.

[BibT_eX]

[DOI]

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021

Allocating Large Vocabulary Capacity for Cross-Lingual Language Model Pre-Training.

[BibT_eX]

[DOI]

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021

mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs.

[BibT_eX]

[DOI]

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021

Zero-Shot Cross-Lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders.

[BibT_eX]

[DOI]

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021

Consistency Regularization for Cross-Lingual Fine-Tuning.

[BibT_eX]

[DOI]

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021

Memory-Efficient Differentiable Transformer Architecture Search.

[BibT_eX]

[DOI]

Proceedings of the Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, 2021

Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains.

[BibT_eX]

[DOI]

Proceedings of the Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, 2021

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers.

[BibT_eX]

[DOI]

Proceedings of the Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, 2021

Learning to Sample Replacements for ELECTRA Pre-Training.

[BibT_eX]

[DOI]

Proceedings of the Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, 2021

Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment.

[BibT_eX]

[DOI]

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 2021

Self-Attention Attribution: Interpreting Information Interactions Inside Transformer.

[BibT_eX]

[DOI]

Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021

2020

XLM-T: Scaling up Multilingual Machine Translation with Pretrained Cross-lingual Transformer Encoders.

[BibT_eX]

[DOI]

CoRR, 2020

MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers.

[BibT_eX]

[DOI]

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

Investigating Learning Dynamics of BERT Fine-Tuning.

[BibT_eX]

[DOI]

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, 2020

Can Monolingual Pretrained Models Help Cross-Lingual Classification?

[BibT_eX]

[DOI]

Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, 2020

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training.

[BibT_eX]

[DOI]

Proceedings of the 37th International Conference on Machine Learning, 2020

Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks.

[BibT_eX]

[DOI]

Proceedings of the Computer Vision - ECCV 2020, 2020

Harvesting and Refining Question-Answer Pairs for Unsupervised QA.

[BibT_eX]

[DOI]

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020

Cross-Lingual Natural Language Generation via Pre-Training.

[BibT_eX]

[DOI]

Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020

2019

Multitask learning for biomedical named entity recognition with cross-sharing structure.

[BibT_eX]

[DOI]

BMC Bioinform., 2019

Unified Language Model Pre-training for Natural Language Understanding and Generation.

[BibT_eX]

[DOI]

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

Visualizing and Understanding the Effectiveness of BERT.

[BibT_eX]

[DOI]

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019

Learning to Ask Unanswerable Questions for Machine Reading Comprehension.

[BibT_eX]

[DOI]

Proceedings of the 57th Conference of the Association for Computational Linguistics, 2019

Data-to-text Generation with Entity Modeling.

[BibT_eX]

[DOI]

Ratish Puduppully

Li Dong

Mirella Lapata

Proceedings of the 57th Conference of the Association for Computational Linguistics, 2019

Inspecting Unification of Encoding and Matching with Transformer: A Case Study of Machine Reading Comprehension.

[BibT_eX]

[DOI]

Proceedings of the 2nd Workshop on Machine Reading for Question Answering, 2019

Data-to-Text Generation with Content Selection and Planning.

[BibT_eX]

[DOI]

Ratish Puduppully

Li Dong

Mirella Lapata

Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, 2019

2018

Proactive Resource Management for LTE in Unlicensed Spectrum: A Deep Learning Perspective.

[BibT_eX]

[DOI]

Ursula Challita

Li Dong

Walid Saad

IEEE Trans. Wirel. Commun., 2018

Confidence Modeling for Neural Semantic Parsing.

[BibT_eX]

[DOI]

Li Dong

Chris Quirk

Mirella Lapata

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018

Coarse-to-Fine Decoding for Neural Semantic Parsing.

[BibT_eX]

[DOI]

Li Dong

Mirella Lapata

Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 2018

2017

Proactive Resource Management in LTE-U Systems: A Deep Learning Perspective.

[BibT_eX]

[DOI]

Ursula Challita

Li Dong

Walid Saad

CoRR, 2017

Learning to Paraphrase for Question Answering.

[BibT_eX]

[DOI]

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017

Learning to Generate Product Reviews from Attributes.

[BibT_eX]

[DOI]

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, 2017

2016

Adaptive Multi-Compositionality for Recursive Neural Network Models.

[BibT_eX]

[DOI]

IEEE ACM Trans. Audio Speech Lang. Process., 2016

Unsupervised Word and Dependency Path Embeddings for Aspect Term Extraction.

[BibT_eX]

[DOI]

Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, 2016

Solving and Generating Chinese Character Riddles.

[BibT_eX]

[DOI]

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016

Long Short-Term Memory-Networks for Machine Reading.

[BibT_eX]

[DOI]

Jianpeng Cheng

Li Dong

Mirella Lapata

Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016

Language to Logical Form with Neural Attention.

[BibT_eX]

[DOI]

Li Dong

Mirella Lapata

Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, 2016

2015

A Joint Segmentation and Classification Framework for Sentence Level Sentiment Classification.

[BibT_eX]

[DOI]

IEEE ACM Trans. Audio Speech Lang. Process., 2015

A Statistical Parsing Framework for Sentiment Classification.

[BibT_eX]

[DOI]

Comput. Linguistics, 2015

Splusplus: A Feature-Rich Two-stage Classifier for Sentiment Analysis of Tweets.

[BibT_eX]

[DOI]

Proceedings of the 9th International Workshop on Semantic Evaluation, 2015

A Hybrid Neural Model for Type Classification of Entity Mentions.

[BibT_eX]

[DOI]

Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, 2015

Question Answering over Freebase with Multi-Column Convolutional Neural Networks.

[BibT_eX]

[DOI]

Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, 2015

Ranking with Recursive Neural Networks and Its Application to Multi-Document Summarization.

[BibT_eX]

[DOI]

Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015

2014

A Joint Segmentation and Classification Framework for Sentiment Analysis.

[BibT_eX]