Lechao Xiao

According to our database, Lechao Xiao authored at least 25 papers between 2018 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2024

Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models.

Avi Singh, John D. Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Xavier Garcia, Peter J. Liu, James Harrison, Jaehoon Lee, Kelvin Xu, Aaron T. Parisi, Abhishek Kumar, Alexander A. Alemi, Alex Rizkowsky, Azade Nova, Ben Adlam, Bernd Bohnet, Gamaleldin Fathy Elsayed, Hanie Sedghi, Igor Mordatch, Isabelle Simpson, Izzeddin Gur, Jasper Snoek, Jeffrey Pennington, Jiri Hron, Kathleen Kenealy, Kevin Swersky, Kshiteej Mahajan, Laura Culp, Lechao Xiao, Maxwell L. Bileschi, Noah Constant, Roman Novak, Rosanne Liu, Tris Warkentin, Yundi Qian, Yamini Bansal, Ethan Dyer, Behnam Neyshabur, Jascha Sohl-Dickstein, Noah Fiedel

Trans. Mach. Learn. Res., 2024

Rethinking Conventional Wisdom in Machine Learning: From Generalization to Scaling.

Lechao Xiao

CoRR, 2024

Training Language Models on the Knowledge Graph: Insights on Hallucinations and Their Detectability.

Jiri Hron, Laura Culp, Gamaleldin F. Elsayed, Rosanne Liu, Ben Adlam, Maxwell L. Bileschi, Bernd Bohnet, JD Co-Reyes, Noah Fiedel, C. Daniel Freeman, Izzeddin Gur, Kathleen Kenealy, Jaehoon Lee, Peter J. Liu, Gaurav Mishra, Igor Mordatch, Azade Nova, Roman Novak, Aaron Parisi, Jeffrey Pennington, Alex Rizkowsky, Isabelle Simpson, Hanie Sedghi, Jascha Sohl-Dickstein, Kevin Swersky, Sharad Vikram, Tris Warkentin, Lechao Xiao, Kelvin Xu, Jasper Snoek, Simon Kornblith

CoRR, 2024

4+3 Phases of Compute-Optimal Neural Scaling Laws.

Elliot Paquette, Courtney Paquette, Lechao Xiao, Jeffrey Pennington

CoRR, 2024

Scaling Exponents Across Parameterizations and Optimizers.

Katie E. Everett, Lechao Xiao, Mitchell Wortsman, Alexander A. Alemi, Roman Novak, Peter J. Liu, Izzeddin Gur, Jascha Sohl-Dickstein, Leslie Pack Kaelbling, Jaehoon Lee, Jeffrey Pennington

Proceedings of the Forty-first International Conference on Machine Learning, 2024

Small-scale proxies for large-scale Transformer training instabilities.

Mitchell Wortsman, Peter J. Liu, Lechao Xiao, Katie E. Everett, Alexander A. Alemi, Ben Adlam, John D. Co-Reyes, Izzeddin Gur, Abhishek Kumar, Roman Novak, Jeffrey Pennington, Jascha Sohl-Dickstein, Kelvin Xu, Jaehoon Lee, Justin Gilmer, Simon Kornblith

Proceedings of the Twelfth International Conference on Learning Representations, 2024

2023

Frontier Language Models are not Robust to Adversarial Arithmetic, or "What do I need to say so you agree 2+2=5?

C. Daniel Freeman, Laura Culp, Aaron Parisi, Maxwell L. Bileschi, Gamaleldin F. Elsayed, Alex Rizkowsky, Isabelle Simpson, Alex Alemi, Azade Nova, Ben Adlam, Bernd Bohnet, Gaurav Mishra, Hanie Sedghi, Igor Mordatch, Izzeddin Gur, Jaehoon Lee, John D. Co-Reyes, Jeffrey Pennington, Kelvin Xu, Kevin Swersky, Kshiteej Mahajan, Lechao Xiao, Rosanne Liu, Simon Kornblith, Noah Constant, Peter J. Liu, Roman Novak, Yundi Qian, Noah Fiedel, Jascha Sohl-Dickstein

CoRR, 2023

2022

Precise Learning Curves and Higher-Order Scaling Limits for Dot Product Kernel Regression.

Lechao Xiao, Jeffrey Pennington

CoRR, 2022

Precise Learning Curves and Higher-Order Scalings for Dot-product Kernel Regression.

Lechao Xiao, Hong Hu, Theodor Misiakiewicz, Yue Lu, Jeffrey Pennington

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Fast Neural Kernel Embeddings for General Activations.

Insu Han, Amir Zandieh, Jaehoon Lee, Roman Novak, Lechao Xiao, Amin Karbasi

Proceedings of the Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, 2022

Synergy and Symmetry in Deep Learning: Interactions between the Data, Model, and Inference Algorithm.

Lechao Xiao, Jeffrey Pennington

Proceedings of the International Conference on Machine Learning, 2022

Eigenspace Restructuring: A Principle of Space and Frequency in Neural Networks.

Lechao Xiao

Proceedings of the Conference on Learning Theory, 2-5 July 2022, London, UK., 2022

2021

Dataset Distillation with Infinitely Wide Convolutional Networks.

Timothy Nguyen, Roman Novak, Lechao Xiao, Jaehoon Lee

Proceedings of the Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, 2021

Exploring the Uncertainty Properties of Neural Networks' Implicit Priors in the Infinite-Width Limit.

Ben Adlam, Jaehoon Lee, Lechao Xiao, Jeffrey Pennington, Jasper Snoek

Proceedings of the 9th International Conference on Learning Representations, 2021

2020

Finite Versus Infinite Neural Networks: an Empirical Study.

Jaehoon Lee, Samuel S. Schoenholz, Jeffrey Pennington, Ben Adlam, Lechao Xiao, Roman Novak, Jascha Sohl-Dickstein

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networks.

Wei Hu, Lechao Xiao, Ben Adlam, Jeffrey Pennington

Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, 2020

Disentangling Trainability and Generalization in Deep Neural Networks.

Lechao Xiao, Jeffrey Pennington, Samuel Stern Schoenholz

Proceedings of the 37th International Conference on Machine Learning, 2020

Neural Tangents: Fast and Easy Infinite Neural Networks in Python.

Roman Novak, Lechao Xiao, Jiri Hron, Jaehoon Lee, Alexander A. Alemi, Jascha Sohl-Dickstein, Samuel S. Schoenholz

Proceedings of the 8th International Conference on Learning Representations, 2020

Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networks.

Wei Hu, Lechao Xiao, Jeffrey Pennington

Proceedings of the 8th International Conference on Learning Representations, 2020

2019

Disentangling trainability and generalization in deep learning.

Lechao Xiao, Jeffrey Pennington, Samuel S. Schoenholz

CoRR, 2019

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent.

Jaehoon Lee, Lechao Xiao, Samuel S. Schoenholz, Yasaman Bahri, Jascha Sohl-Dickstein, Jeffrey Pennington

CoRR, 2019

Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent.

Jaehoon Lee, Lechao Xiao, Samuel S. Schoenholz, Yasaman Bahri, Roman Novak, Jascha Sohl-Dickstein, Jeffrey Pennington

Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, 2019

Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes.

Roman Novak, Lechao Xiao, Yasaman Bahri, Jaehoon Lee, Greg Yang, Jiri Hron, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-Dickstein

Proceedings of the 7th International Conference on Learning Representations, 2019

2018

Bayesian Convolutional Neural Networks with Many Channels are Gaussian Processes.

Roman Novak, Lechao Xiao, Jaehoon Lee, Yasaman Bahri, Daniel A. Abolafia, Jeffrey Pennington, Jascha Sohl-Dickstein

CoRR, 2018

Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10, 000-Layer Vanilla Convolutional Neural Networks.

Lechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein, Samuel S. Schoenholz, Jeffrey Pennington

Proceedings of the 35th International Conference on Machine Learning, 2018