Learning Term Embeddings for Taxonomic Relation Identification Using Dynamic Weighting Neural Network. - Anh Tuan Luu; Yi Tay; Siu Cheung Hui; See-Kiong Ng (EMNLP, 2016)

Cross Device Matching for Online Advertising with Neural Feature Ensembles : First Place Solution at CIKM Cup 2016. - Yi Tay; Cong-Minh Phan; Tuan-Anh Nguyen Pham (CoRR, 2016)

Non-Parametric Estimation of Multiple Embeddings for Link Prediction on Dynamic Knowledge Graphs. - Yi Tay; Anh Tuan Luu; Siu Cheung Hui (AAAI, 2017)

NeuPL: Attention-based Semantic Matching and Pair-Linking for Entity Disambiguation. - Minh C. Phan; Aixin Sun; Yi Tay; Jialong Han; Chenliang Li (CIKM, 2017)

Dyadic Memory Networks for Aspect-based Sentiment Analysis. - Yi Tay; Luu Anh Tuan; Siu Cheung Hui (CIKM, 2017)

Multi-Task Neural Network for Non-discrete Attribute Prediction in Knowledge Graphs. - Yi Tay; Luu Anh Tuan; Minh C. Phan; Siu Cheung Hui (CIKM, 2017)

Cross-Device User Linking: URL, Session, Visiting Time, and Device-log Embedding. - Minh C. Phan; Aixin Sun; Yi Tay (SIGIR, 2017)

Learning to Rank Question Answer Pairs with Holographic Dual LSTM Architecture. - Yi Tay; Minh C. Phan; Anh Tuan Luu; Siu Cheung Hui (SIGIR, 2017)

Random Semantic Tensor Ensemble for Scalable Knowledge Graph Link Prediction. - Yi Tay; Anh Tuan Luu; Siu Cheung Hui; Falk Brauer (WSDM, 2017)

Translational Recommender Networks. - Yi Tay; Anh Tuan Luu; Siu Cheung Hui (CoRR, 2017)

Enabling Efficient Question Answer Retrieval via Hyperbolic Neural Networks. - Yi Tay; Anh Tuan Luu; Siu Cheung Hui (CoRR, 2017)

Learning to Rank Question Answer Pairs with Holographic Dual LSTM Architecture. - Yi Tay; Cong-Minh Phan; Anh Tuan Luu; Siu Cheung Hui (CoRR, 2017)

SkipFlow: Incorporating Neural Coherence Features for End-to-End Automatic Text Scoring. - Yi Tay; Minh C. Phan; Luu Anh Tuan; Siu Cheung Hui (AAAI, 2018)

Cross Temporal Recurrent Networks for Ranking Question Answer Pairs. - Yi Tay; Luu Anh Tuan; Siu Cheung Hui (AAAI, 2018)

Learning to Attend via Word-Aspect Associative Fusion for Aspect-Based Sentiment Analysis. - Yi Tay; Luu Anh Tuan; Siu Cheung Hui (AAAI, 2018)

Reasoning with Sarcasm by Reading In-Between. - Yi Tay; Anh Tuan Luu; Siu Cheung Hui; Jian Su (ACL (1), 2018)

Compare, Compress and Propagate: Enhancing Neural Architectures with Alignment Factorization for Natural Language Inference. - Yi Tay; Anh Tuan Luu; Siu Cheung Hui (EMNLP, 2018)

Multi-Granular Sequence Encoding via Dilated Compositional Units for Reading Comprehension. - Yi Tay; Anh Tuan Luu; Siu Cheung Hui (EMNLP, 2018)

Co-Stack Residual Affinity Networks with Multi-level Attention Refinement for Matching Text Sequences. - Yi Tay; Anh Tuan Luu; Siu Cheung Hui (EMNLP, 2018)

Attentive Gated Lexicon Reader with Contrastive Contextual Co-Attention for Sentiment Classification. - Yi Tay; Anh Tuan Luu; Siu Cheung Hui; Jian Su (EMNLP, 2018)

CoupleNet: Paying Attention to Couples with Coupled Attention for Relationship Recommendation. - Yi Tay; Luu Anh Tuan; Siu Cheung Hui (ICWSM, 2018)

Hermitian Co-Attention Networks for Text Matching in Asymmetrical Domains. - Yi Tay; Anh Tuan Luu; Siu Cheung Hui (IJCAI, 2018)

Multi-Pointer Co-Attention Networks for Recommendation. - Yi Tay; Anh Tuan Luu; Siu Cheung Hui (KDD, 2018)

Multi-Cast Attention Networks. - Yi Tay; Luu Anh Tuan; Siu Cheung Hui (KDD, 2018)

Recurrently Controlled Recurrent Networks. - Yi Tay; Anh Tuan Luu; Siu Cheung Hui (NeurIPS, 2018)

Densely Connected Attention Propagation for Reading Comprehension. - Yi Tay; Anh Tuan Luu; Siu Cheung Hui; Jian Su (NeurIPS, 2018)

Hyperbolic Representation Learning for Fast and Efficient Neural Question Answering. - Yi Tay; Luu Anh Tuan; Siu Cheung Hui (WSDM, 2018)

Latent Relational Metric Learning via Memory-based Attention for Collaborative Ranking. - Yi Tay; Luu Anh Tuan; Siu Cheung Hui (WWW, 2018)

A Compare-Propagate Architecture with Alignment Factorization for Natural Language Inference. - Yi Tay; Luu Anh Tuan; Siu Cheung Hui (CoRR, 2018)

Multi-range Reasoning for Machine Comprehension. - Yi Tay; Luu Anh Tuan; Siu Cheung Hui (CoRR, 2018)

Multi-Cast Attention Networks for Retrieval-based Question Answering and Response Prediction. - Yi Tay; Luu Anh Tuan; Siu Cheung Hui (CoRR, 2018)

Next Item Recommendation with Self-Attention. - Shuai Zhang 007; Yi Tay; Lina Yao 001; Aixin Sun (CoRR, 2018)

Hyperbolic Recommender Systems. - Tran Dang Quang Vinh; Yi Tay; Shuai Zhang 007; Gao Cong; Xiao-Li Li 001 (CoRR, 2018)

Holographic Factorization Machines for Recommendation. - Yi Tay; Shuai Zhang 007; Anh Tuan Luu; Siu Cheung Hui; Lina Yao 001; Tran Dang Quang Vinh (AAAI, 2019)

Robust Representation Learning of Biomedical Names. - Minh C. Phan; Aixin Sun; Yi Tay (ACL (1), 2019)

Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives. - Yi Tay; Shuohang Wang; Anh Tuan Luu; Jie Fu 001; Minh C. Phan; Xingdi Yuan; Jinfeng Rao; Siu Cheung Hui; Aston Zhang (ACL (1), 2019)

Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks. - Yi Tay; Aston Zhang; Anh Tuan Luu; Jinfeng Rao; Shuai Zhang 007; Shuohang Wang; Jie Fu 001; Siu Cheung Hui (ACL (1), 2019)

Confusionset-guided Pointer Networks for Chinese Spelling Check. - Dingmin Wang; Yi Tay; Li Zhong (ACL (1), 2019)

Bridging the Gap between Relevance Matching and Semantic Matching for Short Text Similarity Modeling. - Jinfeng Rao; Linqing Liu; Yi Tay; Hsiu-Wei Yang; Peng Shi 010; Jimmy Lin (EMNLP/IJCNLP (1), 2019)

DeepRec: An Open-source Toolkit for Deep Learning based Recommendation. - Shuai Zhang 007; Yi Tay; Lina Yao 001; Bin Wu 019; Aixin Sun (IJCAI, 2019)

Quaternion Collaborative Filtering for Recommendation. - Shuai Zhang 007; Lina Yao 001; Lucas Vinh Tran; Aston Zhang; Yi Tay (IJCAI, 2019)

Quaternion Knowledge Graph Embeddings. - Shuai Zhang 007; Yi Tay; Lina Yao 001; Qi Liu 049 (NeurIPS, 2019)

Compositional De-Attention Networks. - Yi Tay; Anh Tuan Luu; Aston Zhang; Shuohang Wang; Siu Cheung Hui (NeurIPS, 2019)

Interact and Decide: Medley of Sub-Attention Networks for Effective Group Recommendation. - Lucas Vinh Tran; Tuan-Anh Nguyen Pham; Yi Tay; Yiding Liu; Gao Cong; Xiaoli Li 001 (SIGIR, 2019)

Quaternion Knowledge Graph Embedding. - Shuai Zhang 007; Yi Tay; Lina Yao 001; Qi Liu 049 (CoRR, 2019)

Deep Learning Based Recommender System: A Survey and New Perspectives. - Shuai Zhang 007; Lina Yao 001; Aixin Sun; Yi Tay (ACM Comput. Surv., 2019)

Pair-Linking for Collective Entity Disambiguation: Two Could Be Better Than All. - Minh C. Phan; Aixin Sun; Yi Tay; Jialong Han; Chenliang Li (IEEE Trans. Knowl. Data Eng., 2019)

Neural architectures for natural language understanding - Yi Tay (2019)

Multi-Level Head-Wise Match and Aggregation in Transformer for Textual Sequence Matching. - Shuohang Wang; Yunshi Lan; Yi Tay; Jing Jiang 001; Jingjing Liu 001 (AAAI, 2020)

Reverse Engineering Configurations of Neural Text Generation Models. - Yi Tay; Dara Bahri; Che Zheng; Clifford Brunk; Donald Metzler; Andrew Tomkins (ACL, 2020)

Would you Rather? A New Benchmark for Learning Machine Alignment with Cultural Values and Social Preferences. - Yi Tay; Donovan Ong; Jie Fu 001; Alvin Chan; Nancy Chen; Anh Tuan Luu; Chris Pal (ACL, 2020)

Interactive Machine Comprehension with Information Seeking Agents. - Xingdi Yuan; Jie Fu 001; Marc-Alexandre Côté; Yi Tay; Chris Pal; Adam Trischler (ACL, 2020)

What It Thinks Is Important Is Important: Robustness Transfers Through Input Gradients. - Alvin Chan; Yi Tay; Yew-Soon Ong (CVPR, 2020)

Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder. - Alvin Chan; Yi Tay; Yew-Soon Ong; Aston Zhang (EMNLP (Findings), 2020)

Jacobian Adversarially Regularized Networks for Robustness. - Alvin Chan; Yi Tay; Yew-Soon Ong; Jie Fu (ICLR, 2020)

Sparse Sinkhorn Attention. - Yi Tay; Dara Bahri; Liu Yang; Donald Metzler; Da-Cheng Juan (ICML, 2020)

Choppy: Cut Transformer for Ranked List Truncation. - Dara Bahri; Yi Tay; Che Zheng; Donald Metzler; Andrew Tomkins (SIGIR, 2020)

HyperML: A Boosting Metric Learning Approach in Hyperbolic Space for Recommender Systems. - Lucas Vinh Tran; Yi Tay; Shuai Zhang 007; Gao Cong; Xiaoli Li 001 (WSDM, 2020)

HyperGrid: Efficient Multi-Task Transformers with Grid-wise Decomposable Hyper Projections. - Yi Tay; Zhe Zhao 001; Dara Bahri; Donald Metzler; Da-Cheng Juan (CoRR, 2020)

Holistic Multi-Modal Memory Network for Movie Question Answering. - Anran Wang 001; Anh Tuan Luu; Chuan-Sheng Foo; Hongyuan Zhu; Yi Tay; Vijay Chandrasekhar 001 (IEEE Trans. Image Process., 2020)

How Reliable are Model Diagnostics? - Vamsi Aribandi; Yi Tay; Donald Metzler (ACL/IJCNLP (Findings), 2021)

StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling. - Yikang Shen; Yi Tay; Che Zheng; Dara Bahri; Donald Metzler; Aaron C. Courville (ACL/IJCNLP (1), 2021)

Are Pretrained Convolutions Better than Pretrained Transformers? - Yi Tay; Mostafa Dehghani 001; Jai Prakash Gupta 001; Vamsi Aribandi; Dara Bahri; Zhen Qin 001; Donald Metzler (ACL/IJCNLP (1), 2021)

On Orthogonality Constraints for Transformers. - Aston Zhang; Alvin Chan; Yi Tay; Jie Fu 001; Shuohang Wang; Shuai Zhang 007; Huajie Shao; Shuochao Yao; Roy Ka-Wei Lee (ACL/IJCNLP (2), 2021)

Do Transformer Modifications Transfer Across Implementations and Applications? - Sharan Narang; Hyung Won Chung; Yi Tay; Liam Fedus; Thibault Févry; Michael Matena; Karishma Malkan; Noah Fiedel; Noam Shazeer; Zhenzhong Lan; Yanqi Zhou; Wei Li 133; et al. (EMNLP (1), 2021)

Are Neural Rankers still Outperformed by Gradient Boosted Decision Trees? - Zhen Qin 001; Le Yan; Honglei Zhuang; Yi Tay; Rama Kumar Pasumarthi; Xuanhui Wang; Michael Bendersky; Marc Najork (ICLR, 2021)

Long Range Arena : A Benchmark for Efficient Transformers. - Yi Tay; Mostafa Dehghani 001; Samira Abnar; Yikang Shen; Dara Bahri; Philip Pham; Jinfeng Rao; Liu Yang; Sebastian Ruder; Donald Metzler (ICLR, 2021)

HyperGrid Transformers: Towards A Single Model for Multiple Tasks. - Yi Tay; Zhe Zhao 001; Dara Bahri; Donald Metzler; Da-Cheng Juan (ICLR, 2021)

Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameters. - Aston Zhang; Yi Tay; Shuai Zhang 007; Alvin Chan; Anh Tuan Luu; Siu Cheung Hui; Jie Fu 001 (ICLR, 2021)

OmniNet: Omnidirectional Representations from Transformers. - Yi Tay; Mostafa Dehghani 001; Vamsi Aribandi; Jai Prakash Gupta 001; Philip Pham; Zhen Qin 001; Dara Bahri; Da-Cheng Juan; Donald Metzler (ICML, 2021)

Synthesizer: Rethinking Self-Attention for Transformer Models. - Yi Tay; Dara Bahri; Donald Metzler; Da-Cheng Juan; Zhe Zhao 001; Che Zheng (ICML, 2021)

Knowledge Router: Learning Disentangled Representations for Knowledge Graphs. - Shuai Zhang 007; Xi Rao; Yi Tay; Ce Zhang 001 (NAACL-HLT, 2021)

Self-Instantiated Recurrent Units with Dynamic Soft Recursion. - Aston Zhang; Yi Tay; Yikang Shen; Alvin Chan; Shuai Zhang 007 (NeurIPS, 2021)

Generative Models are Unsupervised Predictors of Page Quality: A Colossal-Scale Study. - Dara Bahri; Yi Tay; Che Zheng; Cliff Brunk; Donald Metzler; Andrew Tomkins (WSDM, 2021)

Label Smoothed Embedding Hypothesis for Out-of-Distribution Detection. - Dara Bahri; Heinrich Jiang; Yi Tay; Donald Metzler (CoRR, 2021)

Switch Spaces: Learning Product Spaces with Sparse Gating. - Shuai Zhang 007; Yi Tay; Wenqi Jiang; Da-Cheng Juan; Ce Zhang 001 (CoRR, 2021)

Rethinking Search: Making Experts out of Dilettantes. - Donald Metzler; Yi Tay; Dara Bahri; Marc Najork (CoRR, 2021)

Are Pre-trained Convolutions Better than Pre-trained Transformers? - Yi Tay; Mostafa Dehghani 001; Jai Prakash Gupta 001; Dara Bahri; Vamsi Aribandi; Zhen Qin 001; Donald Metzler (CoRR, 2021)

The Benchmark Lottery. - Mostafa Dehghani 001; Yi Tay; Alexey A. Gritsenko; Zhe Zhao 001; Neil Houlsby; Fernando Diaz 001; Donald Metzler; Oriol Vinyals (CoRR, 2021)

Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers. - Yi Tay; Mostafa Dehghani 001; Jinfeng Rao; William Fedus; Samira Abnar; Hyung Won Chung; Sharan Narang; Dani Yogatama; Ashish Vaswani; Donald Metzler (CoRR, 2021)

Born Again Neural Rankers. - Zhen Qin 001; Le Yan; Yi Tay; Honglei Zhuang; Xuanhui Wang; Michael Bendersky; Marc Najork (CoRR, 2021)

Improving Compositional Generalization with Self-Training for Data-to-Text Generation. - Sanket Vaibhav Mehta; Jinfeng Rao; Yi Tay; Mihir Kale; Ankur Parikh; Hongtao Zhong; Emma Strubell (CoRR, 2021)

PolyViT: Co-training Vision Transformers on Images, Videos and Audio. - Valerii Likhosherstov; Anurag Arnab; Krzysztof Choromanski; Mario Lucic; Yi Tay; Adrian Weller; Mostafa Dehghani 001 (CoRR, 2021)

Rethinking search: making domain experts out of dilettantes. - Donald Metzler; Yi Tay; Dara Bahri; Marc Najork (SIGIR Forum, 2021)

ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference. - Kai Hui 001; Honglei Zhuang; Tao Chen 008; Zhen Qin 001; Jing Lu; Dara Bahri; Ji Ma; Jai Prakash Gupta 001; Cícero Nogueira dos Santos; Yi Tay; Donald Metzler (ACL (Findings), 2022)

Sharpness-Aware Minimization Improves Language Model Generalization. - Dara Bahri; Hossein Mobahi; Yi Tay (ACL (1), 2022)

Improving Compositional Generalization with Self-Training for Data-to-Text Generation. - Sanket Vaibhav Mehta; Jinfeng Rao; Yi Tay; Mihir Kale; Ankur Parikh; Emma Strubell (ACL (1), 2022)

SCENIC: A JAX Library for Computer Vision Research and Beyond. - Mostafa Dehghani 001; Alexey A. Gritsenko; Anurag Arnab; Matthias Minderer; Yi Tay (CVPR, 2022)

Dense Feature Memory Augmented Transformers for COVID-19 Vaccination Search Classification. - Jai Gupta 001; Yi Tay; Chaitanya Kamath; Vinh Tran 002; Donald Metzler; Shailesh Bavadekar; Mimi Sun; Evgeniy Gabrilovich (EMNLP (Industry Track), 2022)

The Efficiency Misnomer. - Mostafa Dehghani 001; Yi Tay; Anurag Arnab; Lucas Beyer; Ashish Vaswani (ICLR, 2022)

ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning. - Vamsi Aribandi; Yi Tay; Tal Schuster; Jinfeng Rao; Huaixiu Steven Zheng; Sanket Vaibhav Mehta; Honglei Zhuang; Vinh Q. Tran 002; Dara Bahri; Jianmo Ni; Jai Prakash Gupta 001; Kai Hui 001; et al. (ICLR, 2022)

Scarf: Self-Supervised Contrastive Learning using Random Feature Corruption. - Dara Bahri; Heinrich Jiang; Yi Tay; Donald Metzler (ICLR, 2022)

Scale Efficiently: Insights from Pretraining and Finetuning Transformers. - Yi Tay; Mostafa Dehghani 001; Jinfeng Rao; William Fedus; Samira Abnar; Hyung Won Chung; Sharan Narang; Dani Yogatama; Ashish Vaswani; Donald Metzler (ICLR, 2022)

Charformer: Fast Character Transformers via Gradient-based Subword Tokenization. - Yi Tay; Vinh Q. Tran 002; Sebastian Ruder; Jai Prakash Gupta 001; Hyung Won Chung; Dara Bahri; Zhen Qin 001; Simon Baumgartner; Cong Yu 001; Donald Metzler (ICLR, 2022)

HyperPrompt: Prompt-based Task-Conditioning of Transformers. - Yun He; Huaixiu Steven Zheng; Yi Tay; Jai Prakash Gupta 001; Yu Du; Vamsi Aribandi; Zhe Zhao 001; YaGuang Li; Zhao Chen; Donald Metzler; Heng-Tze Cheng; Ed H. Chi (ICML, 2022)

A New Generation of Perspective API: Efficient Multilingual Character-level Transformers. - Alyssa Lees; Vinh Q. Tran 002; Yi Tay; Jeffrey Sorensen; Jai Prakash Gupta 001; Donald Metzler; Lucy Vasserman (KDD, 2022)

Confident Adaptive Language Modeling. - Tal Schuster; Adam Fisch; Jai Gupta 001; Mostafa Dehghani 001; Dara Bahri; Vinh Tran 002; Yi Tay; Donald Metzler (NeurIPS, 2022)

Transformer Memory as a Differentiable Search Index. - Yi Tay; Vinh Tran 002; Mostafa Dehghani 001; Jianmo Ni; Dara Bahri; Harsh Mehta; Zhen Qin 001; Kai Hui 001; Zhe Zhao 001; Jai Prakash Gupta 001; Tal Schuster; William W. Cohen; et al. (NeurIPS, 2022)

Unifying Language Learning Paradigms. - Yi Tay; Mostafa Dehghani 001; Vinh Q. Tran 002; Xavier Garcia; Dara Bahri; Tal Schuster; Huaixiu Steven Zheng; Neil Houlsby; Donald Metzler (CoRR, 2022)

Scaling Instruction-Finetuned Language Models. - Hyung Won Chung; Le Hou; Shayne Longpre; Barret Zoph; Yi Tay; William Fedus; Eric Li; Xuezhi Wang 002; Mostafa Dehghani 001; Siddhartha Brahma; Albert Webson; Shixiang Shane Gu; et al. (CoRR, 2022)

Inverse scaling can become U-shaped. - Jason Wei; Yi Tay; Quoc V. Le (CoRR, 2022)

Emergent Abilities of Large Language Models. - Jason Wei; Yi Tay; Rishi Bommasani; Colin Raffel; Barret Zoph; Sebastian Borgeaud; Dani Yogatama; Maarten Bosma; Denny Zhou; Donald Metzler; Ed H. Chi; Tatsunori Hashimoto; et al. (Trans. Mach. Learn. Res., 2022)

Deep Learning for Recommender Systems. - Shuai Zhang 007; Yi Tay; Lina Yao 001; Aixin Sun; Ce Zhang 001 (Recommender Systems Handbook, 2022)

Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them. - Mirac Suzgun; Nathan Scales; Nathanael Schärli; Sebastian Gehrmann; Yi Tay; Hyung Won Chung; Aakanksha Chowdhery; Quoc V. Le; Ed H. Chi; Denny Zhou; Jason Wei (ACL (Findings), 2023)

CoLT5: Faster Long-Range Transformers with Conditional Computation. - Joshua Ainslie; Tao Lei; Michiel de Jong; Santiago Ontañón; Siddhartha Brahma; Yury Zemlyanskiy; David C. Uthus; Mandy Guo; James Lee-Thorp; Yi Tay; Yun-Hsuan Sung; Sumit Sanghai (EMNLP, 2023)

DSI++: Updating Transformer Memory with New Documents. - Sanket Vaibhav Mehta; Jai Gupta 001; Yi Tay; Mostafa Dehghani 001; Vinh Q. Tran 002; Jinfeng Rao; Marc Najork; Emma Strubell; Donald Metzler (EMNLP, 2023)

Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling? - Yi Tay; Mostafa Dehghani 001; Samira Abnar; Hyung Won Chung; William Fedus; Jinfeng Rao; Sharan Narang; Vinh Q. Tran 002; Dani Yogatama; Donald Metzler (EMNLP (Findings), 2023)

Transcending Scaling Laws with 0.1% Extra Compute. - Yi Tay; Jason Wei; Hyung Won Chung; Vinh Q. Tran 002; David R. So; Siamak Shakeri; Xavier Garcia; Huaixiu Steven Zheng; Jinfeng Rao; Aakanksha Chowdhery; Denny Zhou; Donald Metzler; et al. (EMNLP, 2023)

Symbol tuning improves in-context learning in language models. - Jerry W. Wei; Le Hou; Andrew K. Lampinen; Xiangning Chen; Da Huang; Yi Tay; Xinyun Chen; Yifeng Lu; Denny Zhou; Tengyu Ma 001; Quoc V. Le (EMNLP, 2023)

Inverse Scaling Can Become U-Shaped. - Jason Wei; Najoung Kim; Yi Tay; Quoc V. Le (EMNLP, 2023)

UniMax: Fairer and More Effective Language Sampling for Large-Scale Multilingual Pretraining. - Hyung Won Chung; Xavier Garcia; Adam Roberts; Yi Tay; Orhan Firat; Sharan Narang; Noah Constant (ICLR, 2023)

Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints. - Aran Komatsuzaki; Joan Puigcerver; James Lee-Thorp; Carlos Riquelme Ruiz; Basil Mustafa; Joshua Ainslie; Yi Tay; Mostafa Dehghani 001; Neil Houlsby (ICLR, 2023)

Language models are multilingual chain-of-thought reasoners. - Freda Shi; Mirac Suzgun; Markus Freitag; Xuezhi Wang 002; Suraj Srivats; Soroush Vosoughi; Hyung Won Chung; Yi Tay; Sebastian Ruder; Denny Zhou; Dipanjan Das 001; Jason Wei (ICLR, 2023)

Recitation-Augmented Language Models. - Zhiqing Sun; Xuezhi Wang 002; Yi Tay; Yiming Yang; Denny Zhou (ICLR, 2023)

UL2: Unifying Language Learning Paradigms. - Yi Tay; Mostafa Dehghani 001; Vinh Q. Tran 002; Xavier Garcia; Jason Wei; Xuezhi Wang 002; Hyung Won Chung; Dara Bahri; Tal Schuster; Huaixiu Steven Zheng; Denny Zhou; Neil Houlsby; et al. (ICLR, 2023)

Scaling Vision Transformers to 22 Billion Parameters. - Mostafa Dehghani 001; Josip Djolonga; Basil Mustafa; Piotr Padlewski; Jonathan Heek; Justin Gilmer; Andreas Peter Steiner; Mathilde Caron; Robert Geirhos; Ibrahim Alabdulmohsin; Rodolphe Jenatton; Lucas Beyer; et al. (ICML, 2023)

The Flan Collection: Designing Data and Methods for Effective Instruction Tuning. - Shayne Longpre; Le Hou; Tu Vu; Albert Webson; Hyung Won Chung; Yi Tay; Denny Zhou; Quoc V. Le; Barret Zoph; Jason Wei; Adam Roberts (ICML, 2023)

Recommender Systems with Generative Retrieval. - Shashank Rajput; Nikhil Mehta 002; Anima Singh; Raghunandan Hulikal Keshavan; Trung Vu; Lukasz Heldt; Lichan Hong; Yi Tay; Vinh Q. Tran 002; Jonah Samost; Maciej Kula; Ed H. Chi; et al. (NeurIPS, 2023)

Surprise: Result List Truncation via Extreme Value Theory. - Dara Bahri; Che Zheng; Yi Tay; Donald Metzler; Andrew Tomkins (SIGIR, 2023)

Larger language models do in-context learning differently. - Jerry W. Wei; Jason Wei; Yi Tay; Dustin Tran; Albert Webson; Yifeng Lu; Xinyun Chen; Hanxiao Liu; Da Huang; Denny Zhou; Tengyu Ma 001 (CoRR, 2023)

PaLM 2 Technical Report. - Rohan Anil; Andrew M. Dai; Orhan Firat; Melvin Johnson; Dmitry Lepikhin; Alexandre Passos; Siamak Shakeri; Emanuel Taropa; Paige Bailey; Zhifeng Chen; Eric Chu; Jonathan H. Clark; et al. (CoRR, 2023)

PaLI-X: On Scaling up a Multilingual Vision and Language Model. - Xi Chen 071; Josip Djolonga; Piotr Padlewski; Basil Mustafa; Soravit Changpinyo; Jialin Wu; Carlos Riquelme Ruiz; Sebastian Goodman; Xiao Wang 038; Yi Tay; Siamak Shakeri; Mostafa Dehghani 001; et al. (CoRR, 2023)

Efficient Transformers: A Survey. - Yi Tay; Mostafa Dehghani 001; Dara Bahri; Donald Metzler (ACM Comput. Surv., 2023)

PaLM: Scaling Language Modeling with Pathways. - Aakanksha Chowdhery; Sharan Narang; Jacob Devlin; Maarten Bosma; Gaurav Mishra; Adam Roberts; Paul Barham 001; Hyung Won Chung; Charles Sutton; Sebastian Gehrmann; Parker Schuh; Kensen Shi; et al. (J. Mach. Learn. Res., 2023)

PolyViT: Co-training Vision Transformers on Images, Videos and Audio. - Valerii Likhosherstov; Anurag Arnab; Krzysztof Marcin Choromanski; Mario Lucic; Yi Tay; Mostafa Dehghani 001 (Trans. Mach. Learn. Res., 2023)

On Scaling Up a Multilingual Vision and Language Model. - Xi Chen 071; Josip Djolonga; Piotr Padlewski; Basil Mustafa; Soravit Changpinyo; Jialin Wu; Carlos Riquelme Ruiz; Sebastian Goodman; Xiao Wang 038; Yi Tay; Siamak Shakeri; Mostafa Dehghani 001; et al. (CVPR, 2024)

Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models. - Aitor Ormazabal; Che Zheng; Cyprien de Masson d'Autume; Dani Yogatama; Deyu Fu; Donovan Ong; Eric Chen; Eugenie Lamprecht; Hai Pham; Isaac Ong; Kaloyan Aleksiev; Lei Li 039; et al. (CoRR, 2024)

Vibe-Eval: A hard evaluation suite for measuring progress of multimodal language models. - Piotr Padlewski; Max Bain; Matthew Henderson; Zhongkai Zhu; Nishant Relan; Hai Pham; Donovan Ong; Kaloyan Aleksiev; Aitor Ormazabal; Samuel Phua; Ethan Yeo; Eugenie Lamprecht; et al. (CoRR, 2024)

Scaling Instruction-Finetuned Language Models. - Hyung Won Chung; Le Hou; Shayne Longpre; Barret Zoph; Yi Tay; William Fedus; Yunxuan Li; Xuezhi Wang 002; Mostafa Dehghani 001; Siddhartha Brahma; Albert Webson; Shixiang Shane Gu; et al. (J. Mach. Learn. Res., 2024)