Andrew M. Dai

According to our database, Andrew M. Dai authored at least 60 papers between 2011 and 2024.

Collaborative distances:

• Dijkstra number of three.
• Erdős number of three.

2024

Best Practices and Lessons Learned on Synthetic Data for Language Models.

Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, Andrew M. Dai

CoRR, 2024

Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.

Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy P. Lillicrap, Jean-Baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, Ioannis Antonoglou, Rohan Anil, Sebastian Borgeaud, Andrew M. Dai, Katie Millican, Ethan Dyer, Mia Glaese, Thibault Sottiaux, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, James Molloy, Jilin Chen, Michael Isard, Paul Barham, Tom Hennigan, Ross McIlroy, Melvin Johnson, Johan Schalkwyk, Eli Collins, Eliza Rutherford, Erica Moreira, Kareem Ayoub, Megha Goel, Clemens Meyer, Gregory Thornton, Zhen Yang, Henryk Michalewski, Zaheer Abbas, Nathan Schucher, Ankesh Anand, Richard Ives, James Keeling, Karel Lenc, Salem Haykal, Siamak Shakeri, Pranav Shyam, Aakanksha Chowdhery, Roman Ring, Stephen Spencer, Eren Sezener, et al.

CoRR, 2024

2023

PaLM: Scaling Language Modeling with Pathways.

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, Noah Fiedel

J. Mach. Learn. Res., 2023

Gemini: A Family of Highly Capable Multimodal Models.

Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Slav Petrov, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy P. Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael Isard, Paul Ronald Barham, Tom Hennigan, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, Ryan Doherty, Eli Collins, Clemens Meyer, Eliza Rutherford, Erica Moreira, Kareem Ayoub, Megha Goel, George Tucker, Enrique Piqueras, Maxim Krikun, Iain Barr, Nikolay Savinov, Ivo Danihelka, Becca Roelofs, Anaïs White, Anders Andreassen, Tamara von Glehn, Lakshman Yagati, Mehran Kazemi, Lucas Gonzalez, Misha Khalman, Jakub Sygnowski, et al.

CoRR, 2023

Training Socially Aligned Language Models in Simulated Human Society.

Ruibo Liu, Ruixin Yang, Chenyan Jia, Ge Zhang, Denny Zhou, Andrew M. Dai, Diyi Yang, Soroush Vosoughi

CoRR, 2023

PaLM 2 Technical Report.

Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernández Ábrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan A. Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du, Ethan Dyer, Vladimir Feinberg, Fangxiaoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, et al.

CoRR, 2023

MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks.

Weicheng Kuo, A. J. Piergiovanni, Dahun Kim, Xiyang Luo, Benjamin Caine, Wei Li, Abhijit S. Ogale, Luowei Zhou, Andrew M. Dai, Zhifeng Chen, Claire Cui, Anelia Angelova

CoRR, 2023

Order Matters in the Presence of Dataset Imbalance for Multilingual Learning.

Dami Choi, Derrick Xin, Hamid Dadkhahi, Justin Gilmer, Ankush Garg, Orhan Firat, Chih-Kuan Yeh, Andrew M. Dai, Behrooz Ghorbani

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Brainformers: Trading Simplicity for Efficiency.

Yanqi Zhou, Nan Du, Yanping Huang, Daiyi Peng, Chang Lan, Da Huang, Siamak Shakeri, David R. So, Andrew M. Dai, Yifeng Lu, Zhifeng Chen, Quoc V. Le, Claire Cui, James Laudon, Jeff Dean

Proceedings of the International Conference on Machine Learning, 2023

Mind's Eye: Grounded Language Model Reasoning through Simulation.

Ruibo Liu, Jason Wei, Shixiang Shane Gu, Te-Yen Wu, Soroush Vosoughi, Claire Cui, Denny Zhou, Andrew M. Dai

Proceedings of the Eleventh International Conference on Learning Representations, 2023

Massively Multilingual Shallow Fusion with Large Language Models.

Ke Hu, Tara N. Sainath, Bo Li, Nan Du, Yanping Huang, Andrew M. Dai, Yu Zhang, Rodrigo Cabrera, Zhifeng Chen, Trevor Strohman

Proceedings of the IEEE International Conference on Acoustics, 2023

2022

Scaling Instruction-Finetuned Language Models.

Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Y. Zhao, Yanping Huang, Andrew M. Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, Jason Wei

CoRR, 2022

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models.

Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ameet Rahane, Anantharaman S. Iyer, Anders Andreassen, Andrea Santilli, Andreas Stuhlmüller, Andrew M. Dai, Andrew La, Andrew K. Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakas, et al.

CoRR, 2022

Mixture-of-Experts with Expert Choice Routing.

Yanqi Zhou, Tao Lei, Hanxiao Liu, Nan Du, Yanping Huang, Vincent Y. Zhao, Andrew M. Dai, Zhifeng Chen, Quoc V. Le, James Laudon

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts.

Nan Du, Yanping Huang, Andrew M. Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten P. Bosma, Zongwei Zhou, Tao Wang, Yu Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathleen S. Meier-Hellstern, Toju Duke, Lucas Dixon, Kun Zhang, Quoc V. Le, Yonghui Wu, Zhifeng Chen, Claire Cui

Proceedings of the International Conference on Machine Learning, 2022

Finetuned Language Models are Zero-Shot Learners.

Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, Quoc V. Le

Proceedings of the Tenth International Conference on Learning Representations, 2022

2021

Co-training Transformer with Videos and Images Improves Action Recognition.

Bowen Zhang, Jiahui Yu, Christopher Fifty, Wei Han, Andrew M. Dai, Ruoming Pang, Fei Sha

CoRR, 2021

BEDS-Bench: Behavior of EHR-models under Distributional Shift-A Benchmark.

Anand Avati, Martin Seneviratne, Emily Xue, Zhen Xu, Balaji Lakshminarayanan, Andrew M. Dai

CoRR, 2021

Training independent subnetworks for robust prediction.

Marton Havasi, Rodolphe Jenatton, Stanislav Fort, Jeremiah Zhe Liu, Jasper Snoek, Balaji Lakshminarayanan, Andrew Mingbo Dai, Dustin Tran

Proceedings of the 9th International Conference on Learning Representations, 2021

MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records.

Zhen Xu, David R. So, Andrew M. Dai

Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021

2020

Learnability and Complexity of Quantum Samples.

Murphy Yuezhen Niu, Andrew M. Dai, Li Li, Augustus Odena, Zhengli Zhao, Vadim Smelyanskyi, Hartmut Neven, Sergio Boixo

CoRR, 2020

Google COVID-19 Search Trends Symptoms Dataset: Anonymization Process Description (version 1.0).

Shailesh Bavadekar, Andrew M. Dai, John Davis, Damien Desfontaines, Ilya Eckstein, Katie Everett, Alex Fabrikant, Gerardo Flores, Evgeniy Gabrilovich, Krishna Gadepalli, Shane Glass, Rayman Huang, Chaitanya Kamath, Dennis Kraft, Akim Kumok, Hinali Marfatia, Yael Mayer, Benjamin Miller, Adam Pearce, Irippuge Milinda Perera, Venky Ramachandran, Karthik Raman, Thomas Roessler, Izhak Shafran, Tomer Shekel, Charlotte Stanton, Jacob Stimes, Mimi Sun, Gregory Wellenius, Masrour Zoghi

CoRR, 2020

Learning Unstable Dynamical Systems with Time-Weighted Logarithmic Loss.

Kamil Nar, Yuan Xue, Andrew M. Dai

CoRR, 2020

Compositionality and Capacity in Emergent Languages.

Abhinav Gupta, Cinjon Resnick, Jakob N. Foerster, Andrew M. Dai, Kyunghyun Cho

Proceedings of the 5th Workshop on Representation Learning for NLP, 2020

Learning to Select Best Forecast Tasks for Clinical Outcome Prediction.

Yuan Xue, Nan Du, Anne Mottram, Martin Seneviratne, Andrew M. Dai

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

Deep State-Space Generative Model For Correlated Time-to-Event Predictions.

Yuan Xue, Denny Zhou, Nan Du, Andrew M. Dai, Zhen Xu, Kun Zhang, Claire Cui

Proceedings of the KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2020

Flow Contrastive Estimation of Energy-Based Models.

Ruiqi Gao, Erik Nijkamp, Diederik P. Kingma, Zhen Xu, Andrew M. Dai, Ying Nian Wu

Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020

Explaining an increase in predicted risk for clinical alerts.

Michaela Hardt, Alvin Rajkomar, Gerardo Flores, Andrew M. Dai, Michael D. Howell, Greg Corrado, Claire Cui, Moritz Hardt

Proceedings of the ACM CHIL '20: ACM Conference on Health, 2020

Analyzing the role of model uncertainty for electronic health records.

Michael W. Dusenberry, Dustin Tran, Edward Choi, Jonas Kemp, Jeremy Nixon, Ghassen Jerfel, Katherine A. Heller, Andrew M. Dai

Proceedings of the ACM CHIL '20: ACM Conference on Health, 2020

Capacity, Bandwidth, and Compositionality in Emergent Language Learning.

Cinjon Resnick, Abhinav Gupta, Jakob N. Foerster, Andrew M. Dai, Kyunghyun Cho

Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems, 2020

Learning the Graphical Structure of Electronic Health Records with Graph Convolutional Transformer.

Edward Choi, Zhen Xu, Yujia Li, Michael Dusenberry, Gerardo Flores, Emily Xue, Andrew M. Dai

Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, 2020

2019

Natural Questions: a Benchmark for Question Answering Research.

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur P. Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, Slav Petrov

Trans. Assoc. Comput. Linguistics, 2019

Deep Physiological State Space Model for Clinical Forecasting.

Yuan Xue, Denny Zhou, Nan Du, Andrew M. Dai, Zhen Xu, Kun Zhang, Claire Cui

CoRR, 2019

Modelling EHR timeseries by restricting feature interaction.

Kun Zhang, Yuan Xue, Gerardo Flores, Alvin Rajkomar, Claire Cui, Andrew M. Dai

CoRR, 2019

Federated and Differentially Private Learning for Electronic Health Records.

Stephen R. Pfohl, Andrew M. Dai, Katherine A. Heller

CoRR, 2019

Learning an Adaptive Learning Rate Schedule.

Zhen Xu, Andrew M. Dai, Jonas Kemp, Luke Metz

CoRR, 2019

Improved Patient Classification with Language Model Pretraining Over Clinical Notes.

Jonas Kemp, Alvin Rajkomar, Andrew M. Dai

CoRR, 2019

Graph Convolutional Transformer: Learning the Graphical Structure of Electronic Health Records.

Edward Choi, Zhen Xu, Yujia Li, Michael W. Dusenberry, Gerardo Flores, Yuan Xue, Andrew M. Dai

CoRR, 2019

Gmail Smart Compose: Real-Time Assisted Writing.

Mia Xu Chen, Benjamin N. Lee, Gagan Bansal, Yuan Cao, Shuyuan Zhang, Justin Lu, Jackie Tsay, Yinan Wang, Andrew M. Dai, Zhifeng Chen, Timothy Sohn, Yonghui Wu

Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019

Music Transformer: Generating Music with Long-Term Structure.

Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Ian Simon, Curtis Hawthorne, Noam Shazeer, Andrew M. Dai, Matthew D. Hoffman, Monica Dinculescu, Douglas Eck

Proceedings of the 7th International Conference on Learning Representations, 2019

2018

Scalable and accurate deep learning with electronic health records.

Alvin Rajkomar, Eyal Oren, Kai Chen, Andrew M. Dai, Nissan Hajaj, Michaela Hardt, Peter J. Liu, Xiaobing Liu, Jake Marcus, Mimi Sun, Patrik Sundberg, Hector Yee, Kun Zhang, Yi Zhang, Gerardo Flores, Gavin E. Duggan, Jamie Irvine, Quoc Le, Kurt Litsch, Alexander Mossin, Justin Tansuwan, De Wang, James Wexler, Jimbo Wilson, Dana Ludwig, Samuel L. Volchenboum, Katherine Chou, Michael Pearson, Srinivasan Madabushi, Nigam H. Shah, Atul J. Butte, Michael D. Howell, Claire Cui, Gregory S. Corrado, Jeffrey Dean

npj Digit. Medicine, 2018

Reply: metrics to assess machine learning models.

Alvin Rajkomar, Andrew M. Dai, Mimi Sun, Michaela Hardt, Kai Chen, Kathryn Rough, Jeffrey Dean

npj Digit. Medicine, 2018

An Improved Relative Self-Attention Mechanism for Transformer with Application to Music Generation.

Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Curtis Hawthorne, Andrew M. Dai, Matthew D. Hoffman, Douglas Eck

CoRR, 2018

Scalable and accurate deep learning for electronic health records.

Alvin Rajkomar, Eyal Oren, Kai Chen, Andrew M. Dai, Nissan Hajaj, Peter J. Liu, Xiaobing Liu, Mimi Sun, Patrik Sundberg, Hector Yee, Kun Zhang, Gavin E. Duggan, Gerardo Flores, Michaela Hardt, Jamie Irvine, Quoc V. Le, Kurt Litsch, Jake Marcus, Alexander Mossin, Justin Tansuwan, De Wang, James Wexler, Jimbo Wilson, Dana Ludwig, Samuel L. Volchenboum, Katherine Chou, Michael Pearson, Srinivasan Madabushi, Nigam H. Shah, Atul J. Butte, Michael D. Howell, Claire Cui, Greg Corrado, Jeff Dean

CoRR, 2018

Embedding Text in Hyperbolic Spaces.

Bhuwan Dhingra, Christopher J. Shallue, Mohammad Norouzi, Andrew M. Dai, George E. Dahl

Proceedings of the Twelfth Workshop on Graph-Based Methods for Natural Language Processing, 2018

Learning Longer-term Dependencies in RNNs with Auxiliary Losses.

Trieu H. Trinh, Andrew M. Dai, Thang Luong, Quoc V. Le

Proceedings of the 35th International Conference on Machine Learning, 2018

Learning Longer-term Dependencies in RNNs with Auxiliary Losses.

Trieu H. Trinh, Andrew M. Dai, Minh-Thang Luong, Quoc V. Le

Proceedings of the 6th International Conference on Learning Representations, 2018

Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Step.

William Fedus, Mihaela Rosca, Balaji Lakshminarayanan, Andrew M. Dai, Shakir Mohamed, Ian J. Goodfellow

Proceedings of the 6th International Conference on Learning Representations, 2018

MaskGAN: Better Text Generation via Filling in the _______.

William Fedus, Ian J. Goodfellow, Andrew M. Dai

Proceedings of the 6th International Conference on Learning Representations, 2018

AirDialogue: An Environment for Goal-Oriented Dialogue Research.

Wei Wei, Quoc V. Le, Andrew M. Dai, Jia Li

Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31, 2018

Who Said What: Modeling Individual Labelers Improves Classification.

Melody Y. Guan, Varun Gulshan, Andrew M. Dai, Geoffrey E. Hinton

Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, 2018

2017

Adversarial Training Methods for Semi-Supervised Text Classification.

Takeru Miyato, Andrew M. Dai, Ian J. Goodfellow

Proceedings of the 5th International Conference on Learning Representations, 2017

HyperNetworks.

David Ha, Andrew M. Dai, Quoc V. Le

Proceedings of the 5th International Conference on Learning Representations, 2017

2016

Virtual Adversarial Training for Semi-Supervised Text Classification.

Takeru Miyato, Andrew M. Dai, Ian J. Goodfellow

CoRR, 2016

Generating Sentences from a Continuous Space.

Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Józefowicz, Samy Bengio

Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning, 2016

2015

The Supervised Hierarchical Dirichlet Process.

Andrew M. Dai, Amos J. Storkey

IEEE Trans. Pattern Anal. Mach. Intell., 2015

Document Embedding with Paragraph Vectors.

Andrew M. Dai, Christopher Olah, Quoc V. Le

CoRR, 2015

Semi-supervised Sequence Learning.

Andrew M. Dai, Quoc V. Le

Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, 2015

2011

The Grouped Author-Topic Model for Unsupervised Entity Resolution.

Andrew M. Dai, Amos J. Storkey

Proceedings of the Artificial Neural Networks and Machine Learning - ICANN 2011, 2011

Language-independent compound splitting with morphological operations.

Klaus Macherey, Andrew M. Dai, David Talbot, Ashok C. Popat, Franz Josef Och

Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 2011