Deepak Narayanan

According to our database, Deepak Narayanan authored at least 36 papers between 2016 and 2024.

Collaborative distances:

• Dijkstra number of three.
• Erdős number of three.

2024

Nemotron-4 340B Technical Report.

Bo Adler, Niket Agarwal, Ashwath Aithal, Dong H. Anh, Pallab Bhattacharya, Annika Brundyn, Jared Casper, Bryan Catanzaro, Sharon Clay, Jonathan M. Cohen, Sirshak Das, Ayush Dattagupta, Olivier Delalleau, Leon Derczynski, Yi Dong, Daniel Egert, Ellie Evans, Aleksander Ficek, Denys Fridman, Shaona Ghosh, Boris Ginsburg, Igor Gitman, Tomasz Grzegorzek, Robert Hero, Jining Huang, Vibhu Jawa, Joseph Jennings, Aastha Jhunjhunwala, John Kamalu, Sadaf Khan, Oleksii Kuchaiev, Patrick LeGresley, Hui Li, Jiwei Liu, Zihan Liu, Eileen Long, Ameya Sunil Mahabaleshwarkar, Somshubra Majumdar, James Maki, Miguel Martinez, Maer Rodrigues de Melo, Ivan Moshkov, Deepak Narayanan, Sean Narenthiran, Jesus Navarro, Phong Nguyen, Osvald Nitski, Vahid Noroozi, Guruprasad Nutheti, Christopher Parisien, Jupinder Parmar, Mostofa Patwary, Krzysztof Pawelec, Wei Ping, Shrimai Prabhumoye, Rajarshi Roy, Trisha Saar, Vasanth Rao Naik Sabavat, Sanjeev Satheesh, Jane Polak Scowcroft, Jason Sewall, Pavel Shamis, Gerald Shen, Mohammad Shoeybi, Dave Sizer, Misha Smelyanskiy, Felipe Soares, Makesh Narsimhan Sreedhar, Dan Su, Sandeep Subramanian, Shengyang Sun, Shubham Toshniwal, Hao Wang, Zhilin Wang, Jiaxuan You, Jiaqi Zeng, Jimmy Zhang, Jing Zhang, Vivienne Zhang, Yian Zhang, Chen Zhu

CoRR, 2024

An Empirical Study of Mamba-based Language Models.

Roger Waleffe, Wonmin Byeon, Duncan Riach, Brandon Norick, Vijay Korthikanti, Tri Dao, Albert Gu, Ali Hatamizadeh, Sudhakar Singh, Deepak Narayanan, Garvit Kulshreshtha, Vartika Singh, Jared Casper, Jan Kautz, Mohammad Shoeybi, Bryan Catanzaro

CoRR, 2024

Nemotron-4 15B Technical Report.

Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Mostofa Patwary, Sandeep Subramanian, Dan Su, Chen Zhu, Deepak Narayanan, Aastha Jhunjhunwala, Ayush Dattagupta, Vibhu Jawa, Jiwei Liu, Ameya Mahabaleshwarkar, Osvald Nitski, Annika Brundyn, James Maki, Miguel Martinez, Jiaxuan You, John Kamalu, Patrick LeGresley, Denys Fridman, Jared Casper, Ashwath Aithal, Oleksii Kuchaiev, Mohammad Shoeybi, Jonathan M. Cohen, Bryan Catanzaro

CoRR, 2024

The Case for Co-Designing Model Architectures with Hardware.

Quentin Anthony, Jacob Hatef, Deepak Narayanan, Stella Biderman, Stas Bekman, Junqi Yin, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda

Proceedings of the 53rd International Conference on Parallel Processing, 2024

MGit: A Model Versioning and Management System.

Wei Hao, Daniel Mendoza, Rafael Mendes, Deepak Narayanan, Amar Phanishayee, Asaf Cidon, Junfeng Yang

Proceedings of the Forty-first International Conference on Machine Learning, 2024

2023

Holistic Evaluation of Language Models.

Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel J. Orr, Lucia Zheng, Mert Yüksekgönül, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S. Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, Yuta Koreeda

Trans. Mach. Learn. Res., 2023

Packrat: Automatic Reconfiguration for Latency Minimization in CPU-based DNN Serving.

Ankit Bhardwaj, Amar Phanishayee, Deepak Narayanan, Mihail Tarta, Ryan Stutsman

CoRR, 2023

MGit: A Model Versioning and Management System.

Wei Hao, Daniel Mendoza, Rafael da Silva, Deepak Narayanan, Amar Phanishayee

CoRR, 2023

Cheaply Evaluating Inference Efficiency Metrics for Autoregressive Transformer APIs.

Deepak Narayanan, Keshav Santhanam, Peter Henderson, Rishi Bommasani, Tony Lee, Percy Liang

CoRR, 2023

Kerveros: Efficient and Scalable Cloud Admission Control.

Sultan Mahmud Sajal, Luke Marshall, Beibin Li, Shandan Zhou, Abhisek Pan, Konstantina Mellou, Deepak Narayanan, Timothy Zhu, David Dion, Thomas Moscibroda, Ishai Menache

Proceedings of the 17th USENIX Symposium on Operating Systems Design and Implementation, 2023

Cheaply Estimating Inference Efficiency Metrics for Autoregressive Transformer Models.

Deepak Narayanan, Keshav Santhanam, Peter Henderson, Rishi Bommasani, Tony Lee, Percy Liang

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

Holistic Evaluation of Text-to-Image Models.

Tony Lee, Michihiro Yasunaga, Chenlin Meng, Yifan Mai, Joon Sung Park, Agrim Gupta, Yunzhi Zhang, Deepak Narayanan, Hannah Teufel, Marco Bellagente, Minguk Kang, Taesung Park, Jure Leskovec, Jun-Yan Zhu, Fei-Fei Li, Jiajun Wu, Stefano Ermon, Percy Liang

Proceedings of the Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, 2023

MegaBlocks: Efficient Sparse Training with Mixture-of-Experts.

Trevor Gale, Deepak Narayanan, Cliff Young, Matei Zaharia

Proceedings of the Sixth Conference on Machine Learning and Systems, 2023

2022

Allocation of fungible resources via a fast, scalable price discovery method.

Akshay Agrawal, Stephen P. Boyd, Deepak Narayanan, Fiodar Kazhamiaka, Matei Zaharia

Math. Program. Comput., 2022

2021

Resource-efficient execution of deep learning computations.

Deepak Narayanan

PhD thesis, 2021

Don't Give Up on Large Optimization Problems; POP Them!

Deepak Narayanan, Fiodar Kazhamiaka, Firas Abuzaid, Peter Kraft, Matei Zaharia

CoRR, 2021

Efficient Large-Scale Language Model Training on GPU Clusters.

Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vijay Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, Matei Zaharia

CoRR, 2021

Solving Large-Scale Granular Resource Allocation Problems Efficiently with POP.

Deepak Narayanan, Fiodar Kazhamiaka, Firas Abuzaid, Peter Kraft, Akshay Agrawal, Srikanth Kandula, Stephen P. Boyd, Matei Zaharia

Proceedings of the SOSP '21: ACM SIGOPS 28th Symposium on Operating Systems Principles, 2021

Efficient large-scale language model training on GPU clusters using megatron-LM.

Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vijay Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, Matei Zaharia

Proceedings of the International Conference for High Performance Computing, 2021

Piper: Multidimensional Planner for DNN Parallelization.

Jakub Tarnawski, Deepak Narayanan, Amar Phanishayee

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Memory-Efficient Pipeline-Parallel DNN Training.

Deepak Narayanan, Amar Phanishayee, Kaiyu Shi, Xie Chen, Matei Zaharia

Proceedings of the 38th International Conference on Machine Learning, 2021

2020

A Demonstration of Willump: A Statistically-Aware End-to-end Optimizer for Machine Learning Inference.

Peter Kraft, Daniel Kang, Deepak Narayanan, Shoumik Palkar, Peter Bailis, Matei Zaharia

Proc. VLDB Endow., 2020

Offload Annotations: Bringing Heterogeneous Computing to Existing Libraries and Workloads.

Gina Yuan, Shoumik Palkar, Deepak Narayanan, Matei Zaharia

Proceedings of the 2020 USENIX Annual Technical Conference, 2020

Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads.

Deepak Narayanan, Keshav Santhanam, Fiodar Kazhamiaka, Amar Phanishayee, Matei Zaharia

Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation, 2020

MLPerf Training Benchmark.

Peter Mattson, Christine Cheng, Gregory F. Diamos, Cody Coleman, Paulius Micikevicius, David A. Patterson, Hanlin Tang, Gu-Yeon Wei, Peter Bailis, Victor Bittorf, David Brooks, Dehao Chen, Debo Dutta, Udit Gupta, Kim M. Hazelwood, Andy Hock, Xinyuan Huang, Daniel Kang, David Kanter, Naveen Kumar, Jeffery Liao, Deepak Narayanan, Tayo Oguntebi, Gennady Pekhimenko, Lillian Pentecost, Vijay Janapa Reddi, Taylor Robie, Tom St. John, Carole-Jean Wu, Lingjie Xu, Cliff Young, Matei Zaharia

Proceedings of the Third Conference on Machine Learning and Systems, 2020

Willump: A Statistically-Aware End-to-end Optimizer for Machine Learning Inference.

Peter Kraft, Daniel Kang, Deepak Narayanan, Shoumik Palkar, Peter Bailis, Matei Zaharia

Proceedings of the Third Conference on Machine Learning and Systems, 2020

2019

Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark.

Cody Coleman, Daniel Kang, Deepak Narayanan, Luigi Nardi, Tian Zhao, Jian Zhang, Peter Bailis, Kunle Olukotun, Christopher Ré, Matei Zaharia

ACM SIGOPS Oper. Syst. Rev., 2019

MLPerf Training Benchmark.

Peter Mattson, Christine Cheng, Cody Coleman, Greg Diamos, Paulius Micikevicius, David A. Patterson, Hanlin Tang, Gu-Yeon Wei, Peter Bailis, Victor Bittorf, David Brooks, Dehao Chen, Debojyoti Dutta, Udit Gupta, Kim M. Hazelwood, Andrew Hock, Xinyuan Huang, Bill Jia, Daniel Kang, David Kanter, Naveen Kumar, Jeffery Liao, Guokai Ma, Deepak Narayanan, Tayo Oguntebi, Gennady Pekhimenko, Lillian Pentecost, Vijay Janapa Reddi, Taylor Robie, Tom St. John, Carole-Jean Wu, Lingjie Xu, Cliff Young, Matei Zaharia

CoRR, 2019

PipeDream: generalized pipeline parallelism for DNN training.

Deepak Narayanan, Aaron Harlap, Amar Phanishayee, Vivek Seshadri, Nikhil R. Devanur, Gregory R. Ganger, Phillip B. Gibbons, Matei Zaharia

Proceedings of the 27th ACM Symposium on Operating Systems Principles, 2019

2018

MacroBase: Prioritizing Attention in Fast Data.

Firas Abuzaid, Peter Bailis, Jialin Ding, Edward Gan, Samuel Madden, Deepak Narayanan, Kexin Rong, Sahaana Suri

ACM Trans. Database Syst., 2018

Evaluating End-to-End Optimization for Data Analytics Applications in Weld.

Shoumik Palkar, James Thomas, Deepak Narayanan, Pratiksha Thaker, Rahul Palamuttam, Parimarjan Negi, Anil Shanbhag, Malte Schwarzkopf, Holger Pirk, Saman P. Amarasinghe, Samuel Madden, Matei Zaharia

Proc. VLDB Endow., 2018

PipeDream: Fast and Efficient Pipeline Parallel DNN Training.

Aaron Harlap, Deepak Narayanan, Amar Phanishayee, Vivek Seshadri, Nikhil R. Devanur, Gregory R. Ganger, Phillip B. Gibbons

CoRR, 2018

2017

Weld: Rethinking the Interface Between Data-Intensive Applications.

Shoumik Palkar, James Thomas, Deepak Narayanan, Anil Shanbhag, Rahul Palamuttam, Holger Pirk, Malte Schwarzkopf, Saman P. Amarasinghe, Samuel Madden, Matei Zaharia

CoRR, 2017

MacroBase: Prioritizing Attention in Fast Data.

Peter Bailis, Edward Gan, Samuel Madden, Deepak Narayanan, Kexin Rong, Sahaana Suri

Proceedings of the 2017 ACM International Conference on Management of Data, 2017

A Common Runtime for High Performance Data Analysis.

Shoumik Palkar, James Thomas, Anil Shanbhag, Deepak Narayanan, Holger Pirk, Malte Schwarzkopf, Saman P. Amarasinghe, Matei Zaharia

Proceedings of the 8th Biennial Conference on Innovative Data Systems Research, 2017

2016

MacroBase: Analytic Monitoring for the Internet of Things.

Peter Bailis, Deepak Narayanan, Samuel Madden

CoRR, 2016