Joel Hestness

According to our database, Joel Hestness authored at least 28 papers between 2009 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers.

Gavia Gray, Aman Tiwari, Shane Bergsma, Joel Hestness

CoRR, 2024

Bilingual Adaptation of Monolingual Foundation Models.

Gurpreet Gosal, Yishi Xu, Gokul Ramakrishnan, Rituraj Joshi, Avraham Sheinin, Zhiming Chen, Biswajit Mishra, Natalia Vassilieva, Joel Hestness, Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Onkar Pandit, Satheesh Katipomu, Samta Kamboj, Samujjwal Ghosh, Rahul Pal, Parvez Mullah, Soundar Doraiswamy, Mohamed El Karim Chami, Preslav Nakov

CoRR, 2024

Sparse maximal update parameterization: A holistic approach to sparse training dynamics.

Nolan Dey, Shane Bergsma, Joel Hestness

CoRR, 2024

MediSwift: Efficient Sparse Pre-trained Biomedical Language Models.

Vithursan Thangarasa, Mahmoud Salem, Shreyas Saxena, Kevin Leong, Joel Hestness, Sean Lie

CoRR, 2024

MediSwift: Efficient Sparse Pre-trained Biomedical Language Models.

Vithursan Thangarasa, Mahmoud Salem, Shreyas Saxena, Chen-Yu Leong, Joel Hestness, Sean Lie

Proceedings of the Findings of the Association for Computational Linguistics, 2024

2023

Position Interpolation Improves ALiBi Extrapolation.

Faisal Al-Khateeb, Nolan Dey, Daria Soboleva, Joel Hestness

CoRR, 2023

BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model.

Nolan Dey, Daria Soboleva, Faisal Al-Khateeb, Bowen Yang, Ribhu Pathria, Hemant Khachane, Shaheer Muhammad, Zhiming Chen, Robert Myers, Jacob Robert Steeves, Natalia Vassilieva, Marvin Tom, Joel Hestness

CoRR, 2023

SlimPajama-DC: Understanding Data Combinations for LLM Training.

Zhiqiang Shen, Tianhua Tao, Liqun Ma, Willie Neiswanger, Zhengzhong Liu, Hongyi Wang, Bowen Tan, Joel Hestness, Natalia Vassilieva, Daria Soboleva, Eric P. Xing

CoRR, 2023

Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster.

Nolan Dey, Gurpreet Gosal, Zhiming Chen, Hemant Khachane, William Marshall, Ribhu Pathria, Marvin Tom, Joel Hestness

CoRR, 2023

RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network.

Vitaliy Chiley, Vithursan Thangarasa, Abhay Gupta, Anshul Samar, Joel Hestness, Dennis DeCoste

Proceedings of the Sixth Conference on Machine Learning and Systems, 2023

2022

Time Dependency, Data Flow, and Competitive Advantage.

Ehsan Valavi, Joel Hestness, Marco Iansiti, Newsha Ardalani, Feng Zhu, Karim R. Lakhani

CoRR, 2022

Time and the Value of Data.

Ehsan Valavi, Joel Hestness, Newsha Ardalani, Marco Iansiti

CoRR, 2022

Efficiently Disentangle Causal Representations.

Yuanpeng Li, Joel Hestness, Mohamed Elhoseiny, Liang Zhao, Kenneth Church

CoRR, 2022

2021

Pipelined Backpropagation at Scale: Training Large Models without Batches.

Atli Kosson, Vitaliy Chiley, Abhinav Venigalla, Joel Hestness, Urs Köster

Proceedings of the Fourth Conference on Machine Learning and Systems, 2021

2020

Memory Efficient 3D U-Net with Reversible Mobile Inverted Bottlenecks for Brain Tumor Segmentation.

Mihir Pendse, Vithursan Thangarasa, Vitaliy Chiley, Ryan Holmdahl, Joel Hestness, Dennis DeCoste

Proceedings of the Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, 2020

2019

A survey of 25 years of evaluation.

Kenneth Ward Church, Joel Hestness

Nat. Lang. Eng., 2019

Beyond human-level accuracy: computational challenges in deep learning.

Joel Hestness, Newsha Ardalani, Gregory F. Diamos

Proceedings of the 24th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 2019

Compositional Generalization for Primitive Substitutions.

Yuanpeng Li, Liang Zhao, Jianyu Wang, Joel Hestness

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, 2019

2017

Deep Learning Scaling is Predictable, Empirically.

Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory F. Diamos, Heewoo Jun, Hassan Kianinejad, Md. Mostofa Ali Patwary, Yang Yang, Yanqi Zhou

CoRR, 2017

Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting.

Sercan Ömer Arik, Markus Kliegl, Rewon Child, Joel Hestness, Andrew Gibiansky, Christopher Fougner, Ryan Prenger, Adam Coates

Proceedings of the 18th Annual Conference of the International Speech Communication Association, 2017

2015

gem5-gpu: A Heterogeneous CPU-GPU Simulator.

Jason Power, Joel Hestness, Marc S. Orr, Mark D. Hill, David A. Wood

IEEE Comput. Archit. Lett., 2015

GPU Computing Pipeline Inefficiencies and Optimization Opportunities in Heterogeneous CPU-GPU Processors.

Joel Hestness, Stephen W. Keckler, David A. Wood

Proceedings of the 2015 IEEE International Symposium on Workload Characterization, 2015

2014

A comparative analysis of microarchitecture effects on CPU and GPU memory system behavior.

Joel Hestness, Stephen W. Keckler, David A. Wood

Proceedings of the 2014 IEEE International Symposium on Workload Characterization, 2014

2012

A QoS-Enabled On-Die Interconnect Fabric for Kilo-Node Chips.

Boris Grot, Joel Hestness, Stephen W. Keckler, Onur Mutlu

IEEE Micro, 2012

2011

The gem5 simulator.

Nathan L. Binkert, Bradford M. Beckmann, Gabriel Black, Steven K. Reinhardt, Ali G. Saidi, Arkaprava Basu, Joel Hestness, Derek Hower, Tushar Krishna, Somayeh Sardashti, Rathijit Sen, Korey Sewell, Muhammad Shoaib Bin Altaf, Nilay Vaish, Mark D. Hill, David A. Wood

SIGARCH Comput. Archit. News, 2011

Kilo-NOC: a heterogeneous network-on-chip architecture for scalability and service guarantees.

Boris Grot, Joel Hestness, Stephen W. Keckler, Onur Mutlu

Proceedings of the 38th International Symposium on Computer Architecture (ISCA 2011), 2011

2010

Netrace: dependency-driven trace-based network-on-chip simulation.

Joel Hestness, Boris Grot, Stephen W. Keckler

Proceedings of the Third International Workshop on Network on Chip Architectures, 2010

2009

Express Cube Topologies for on-Chip Interconnects.

Boris Grot, Joel Hestness, Stephen W. Keckler, Onur Mutlu

Proceedings of the 15th International Conference on High-Performance Computer Architecture (HPCA-15 2009), 2009