Kushal Tirumala

According to our database, Kushal Tirumala authored at least 13 papers between 2020 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2024

Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model.

Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, Omer Levy

CoRR, 2024

Brevity is the soul of wit: Pruning long files for code generation.

Aaditya K. Singh, Yu Yang, Kushal Tirumala, Mostafa Elhoushi, Ari S. Morcos

CoRR, 2024

An Introduction to Vision-Language Modeling.

Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay, Alexander C. Li, Adrien Bardes, Suzanne Petryk, Oscar Mañas, Zhiqiu Lin, Anas Mahmoud, Bargav Jayaraman, Mark Ibrahim, Melissa Hall, Yunyang Xiong, Jonathan Lebensold, Candace Ross, Srihari Jayakumar, Chuan Guo, Diane Bouchacourt, Haider Al-Tahan, Karthik Padthe, Vasu Sharma, Hu Xu, Xiaoqing Ellen Tan, Megan Richards, Samuel Lavoie, Pietro Astolfi, Reyhane Askari Hemmat, Jun Chen, Kushal Tirumala, Rim Assouel, Mazda Moayeri, Arjang Talattof, Kamalika Chaudhuri, Zechun Liu, Xilun Chen, Quentin Garrido, Karen Ullrich, Aishwarya Agrawal, Kate Saenko, Asli Celikyilmaz, Vikas Chandra

CoRR, 2024

Text Quality-Based Pruning for Efficient Training of Language Models.

Vasu Sharma, Karthik Padthe, Newsha Ardalani, Kushal Tirumala, Russell Howes, Hu Xu, Po-Yao Huang, Shang-Wen Li, Armen Aghajanyan, Gargi Ghosh, Luke Zettlemoyer

CoRR, 2024

The Unreasonable Ineffectiveness of the Deeper Layers.

Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, Daniel A. Roberts

CoRR, 2024

Effective pruning of web-scale datasets based on complexity of concept clusters.

Amro Abbas, Evgenia Rusak, Kushal Tirumala, Wieland Brendel, Kamalika Chaudhuri, Ari S. Morcos

Proceedings of the Twelfth International Conference on Learning Representations, 2024

2023

Decoding Data Quality via Synthetic Corruptions: Embedding-guided Pruning of Code Data.

Yu Yang, Aaditya K. Singh, Mostafa Elhoushi, Anas Mahmoud, Kushal Tirumala, Fabian Gloeckle, Baptiste Rozière, Carole-Jean Wu, Ari S. Morcos, Newsha Ardalani

CoRR, 2023

SemDeDup: Data-efficient learning at web-scale through semantic deduplication.

Amro Abbas, Kushal Tirumala, Daniel Simig, Surya Ganguli, Ari S. Morcos

CoRR, 2023

D4: Improving LLM Pretraining via Document De-Duplication and Diversification.

Kushal Tirumala, Daniel Simig, Armen Aghajanyan, Ari Morcos

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

2022

Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models.

Kushal Tirumala, Aram H. Markosyan, Luke Zettlemoyer, Armen Aghajanyan

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Investigating Generalization by Controlling Normalized Margin.

Alexander R. Farhang, Jeremy D. Bernstein, Kushal Tirumala, Yang Liu, Yisong Yue

Proceedings of the International Conference on Machine Learning, 2022

Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks.

Tristan Thrush, Kushal Tirumala, Anmol Gupta, Max Bartolo, Pedro Rodriguez, Tariq Kane, William Gaviria Rojas, Peter Mattson, Adina Williams, Douwe Kiela

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 2022

2020

Ensemble Machine Learning Methods for Modeling COVID19 Deaths.

Rahil Bathwal, Pavan Chitta, Kushal Tirumala, Vignesh Varadarajan

CoRR, 2020