Zhaoheng Ni

According to our database, Zhaoheng Ni authored at least 35 papers between 2017 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

Scaling Speech Technology to 1, 000+ Languages.

Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi, Alexei Baevski, Yossi Adi, Xiaohui Zhang, Wei-Ning Hsu, Alexis Conneau, Michael Auli

J. Mach. Learn. Res., 2024

Serialized Speech Information Guidance with Overlapped Encoding Separation for Multi-Speaker Automatic Speech Recognition.

Hao Shi, Yuan Gao, Zhaoheng Ni, Tatsuya Kawahara

CoRR, 2024

High Fidelity Text-Guided Music Generation and Editing via Single-Stage Flow Matching.

Gaël Le Lan, Bowen Shi, Zhaoheng Ni, Sidd Srinivasan, Anurag Kumar, Brian Ellis, David Kant, Varun Nagaraja, Ernie Chang, Wei-Ning Hsu, Yangyang Shi, Vikas Chandra

CoRR, 2024

URGENT Challenge: Universality, Robustness, and Generalizability For Speech Enhancement.

Wangyou Zhang, Robin Scheibler, Kohei Saijo, Samuele Cornell, Chenda Li, Zhaoheng Ni, Anurag Kumar, Jan Pirklbauer, Marvin Sach, Shinji Watanabe, Tim Fingscheidt, Yanmin Qian

CoRR, 2024

Foleygen: Visually-Guided Audio Generation.

Xinhao Mei, Varun Nagaraja, Gaël Le Lan, Zhaoheng Ni, Ernie Chang, Yangyang Shi, Vikas Chandra

Proceedings of the 34th IEEE International Workshop on Machine Learning for Signal Processing, 2024

An Empirical Study on the Impact of Positional Encoding in Transformer-Based Monaural Speech Enhancement.

Qiquan Zhang, Meng Ge, Hongxu Zhu, Eliathamby Ambikairajah, Qi Song, Zhaoheng Ni, Haizhou Li

Proceedings of the IEEE International Conference on Acoustics, 2024

Folding Attention: Memory and Power Optimization for On-Device Transformer-Based Streaming Speech Recognition.

Yang Li, Liangzhen Lai, Yuan Shangguan, Forrest N. Iandola, Zhaoheng Ni, Ernie Chang, Yangyang Shi, Vikas Chandra

Proceedings of the IEEE International Conference on Acoustics, 2024

Stack-and-Delay: A New Codebook Pattern for Music Generation.

Gaël Le Lan, Varun Nagaraja, Ernie Chang, David Kant, Zhaoheng Ni, Yangyang Shi, Forrest N. Iandola, Vikas Chandra

Proceedings of the IEEE International Conference on Acoustics, 2024

Less Peaky and More Accurate CTC Forced Alignment by Label Priors.

Ruizhe Huang, Xiaohui Zhang, Zhaoheng Ni, Li Sun, Moto Hira, Jeff Hwang, Vimal Manohar, Vineel Pratap, Matthew Wiesner, Shinji Watanabe, Daniel Povey, Sanjeev Khudanpur

Proceedings of the IEEE International Conference on Acoustics, 2024

On the Open Prompt Challenge in Conditional Audio Generation.

Ernie Chang, Sidd Srinivasan, Mahi Luthra, Pin-Jie Lin, Varun Nagaraja, Forrest N. Iandola, Zechun Liu, Zhaoheng Ni, Changsheng Zhao, Yangyang Shi, Vikas Chandra

Proceedings of the IEEE International Conference on Acoustics, 2024

2023

Software Design and User Interface of ESPnet-SE++: Speech Enhancement for Robust Speech Processing.

Yen-Ju Lu, Xuankai Chang, Chenda Li, Wangyou Zhang, Samuele Cornell, Zhaoheng Ni, Yoshiki Masuyama, Brian Yan, Robin Scheibler, Zhong-Qiu Wang, Yu Tsao, Yanmin Qian, Shinji Watanabe

J. Open Source Softw., November, 2023

Software Design and User Interface of ESPnet-SE++: Speech Enhancement for Robust Speech Processing (espnet-v.202310).

Yen-Ju Lu, Xuankai Chang, Chenda Li, Wangyou Zhang, Samuele Cornell, Zhaoheng Ni, Yoshiki Masuyama, Brian Yan, Robin Scheibler, Zhong-Qiu Wang, Yu Tsao, Yanmin Qian, Shinji Watanabe

Dataset, October, 2023

A Time-Frequency Attention Module for Neural Speech Enhancement.

Qiquan Zhang, Xinyuan Qian, Zhaoheng Ni, Aaron Nicolson, Eliathamby Ambikairajah, Haizhou Li

IEEE ACM Trans. Audio Speech Lang. Process., 2023

TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch.

Jeff Hwang, Moto Hira, Caroline Chen, Xiaohui Zhang, Zhaoheng Ni, Guangzhi Sun, Pingchuan Ma, Ruizhe Huang, Vineel Pratap, Yuekai Zhang, Anurag Kumar, Chin-Yun Yu, Chuang Zhu, Chunxi Liu, Jacob Kahn, Mirco Ravanelli, Peng Sun, Shinji Watanabe, Yangyang Shi, Yumeng Tao, Robin Scheibler, Samuele Cornell, Sean Kim, Stavros Petridis

CoRR, 2023

Enhance audio generation controllability through representation similarity regularization.

Yangyang Shi, Gaël Le Lan, Varun Nagaraja, Zhaoheng Ni, Xinhao Mei, Ernie Chang, Forrest N. Iandola, Yang Liu, Vikas Chandra

CoRR, 2023

Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Compute.

William Chen, Xuankai Chang, Yifan Peng, Zhaoheng Ni, Soumi Maiti, Shinji Watanabe

Proceedings of the 24th Annual Conference of the International Speech Communication Association, 2023

Ripple Sparse Self-Attention for Monaural Speech Enhancement.

Qiquan Zhang, Hongxu Zhu, Qi Song, Xinyuan Qian, Zhaoheng Ni, Haizhou Li

Proceedings of the IEEE International Conference on Acoustics, 2023

Torchaudio-Squim: Reference-Less Speech Quality and Intelligibility Measures in Torchaudio.

Anurag Kumar, Ke Tan, Zhaoheng Ni, Pranay Manocha, Xiaohui Zhang, Ethan Henderson, Buye Xu

Proceedings of the IEEE International Conference on Acoustics, 2023

TorchAudio 2.1: Advancing Speech Recognition, Self-Supervised Learning, and Audio Processing Components for Pytorch.

Jeff Hwang, Moto Hira, Caroline Chen, Xiaohui Zhang, Zhaoheng Ni, Guangzhi Sun, Pingchuan Ma, Ruizhe Huang, Vineel Pratap, Yuekai Zhang, Anurag Kumar, Chin-Yun Yu, Chuang Zhu, Chunxi Liu, Jacob Kahn, Mirco Ravanelli, Peng Sun, Shinji Watanabe, Yangyang Shi, Yumeng Tao

Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop, 2023

ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit.

Brian Yan, Jiatong Shi, Yun Tang, Hirofumi Inaguma, Yifan Peng, Siddharth Dalmia, Peter Polak, Patrick Fernandes, Dan Berrebbi, Tomoki Hayashi, Xiaohui Zhang, Zhaoheng Ni, Moto Hira, Soumi Maiti, Juan Pino, Shinji Watanabe

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics: System Demonstrations, 2023

2022

ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding.

Yen-Ju Lu, Xuankai Chang, Chenda Li, Wangyou Zhang, Samuele Cornell, Zhaoheng Ni, Yoshiki Masuyama, Brian Yan, Robin Scheibler, Zhong-Qiu Wang, Yu Tsao, Yanmin Qian, Shinji Watanabe

Proceedings of the 23rd Annual Conference of the International Speech Communication Association, 2022

Time-Frequency Attention for Monaural Speech Enhancement.

Qiquan Zhang, Qi Song, Zhaoheng Ni, Aaron Nicolson, Haizhou Li

Proceedings of the IEEE International Conference on Acoustics, 2022

Torchaudio: Building Blocks for Audio and Speech Processing.

Yao-Yuan Yang, Moto Hira, Zhaoheng Ni, Artyom Astafurov, Caroline Chen, Christian Puhrsch, David Pollack, Dmitriy Genzel, Donny Greenberg, Edward Z. Yang, Jason Lian, Jeff Hwang, Ji Chen, Peter Goldsborough, Sean Narenthiran, Shinji Watanabe, Soumith Chintala, Vincent Quenneville-Bélair

Proceedings of the IEEE International Conference on Acoustics, 2022

Towards Low-Distortion Multi-Channel Speech Enhancement: The ESPNET-Se Submission to the L3DAS22 Challenge.

Yen-Ju Lu, Samuele Cornell, Xuankai Chang, Wangyou Zhang, Chenda Li, Zhaoheng Ni, Zhong-Qiu Wang, Shinji Watanabe

Proceedings of the IEEE International Conference on Acoustics, 2022

2021

TorchAudio: Building Blocks for Audio and Speech Processing.

Yao-Yuan Yang, Moto Hira, Zhaoheng Ni, Anjali Chourdia, Artyom Astafurov, Caroline Chen, Ching-Feng Yeh, Christian Puhrsch, David Pollack, Dmitriy Genzel, Donny Greenberg, Edward Z. Yang, Jason Lian, Jay Mahadeokar, Jeff Hwang, Ji Chen, Peter Goldsborough, Prabhat Roy, Sean Narenthiran, Shinji Watanabe, Soumith Chintala, Vincent Quenneville-Bélair, Yangyang Shi

CoRR, 2021

WPD++: An Improved Neural Beamformer for Simultaneous Speech Separation and Dereverberation.

Zhaoheng Ni, Yong Xu, Meng Yu, Bo Wu, Shi-Xiong Zhang, Dong Yu, Michael I. Mandel

Proceedings of the IEEE Spoken Language Technology Workshop, 2021

2020

Combining Spatial Clustering with LSTM Speech Models for Multichannel Speech Enhancement.

Félix Grèzes, Zhaoheng Ni, Viet Anh Trinh, Michael I. Mandel

CoRR, 2020

Improved MVDR Beamforming Using LSTM Speech Models to Clean Spatial Clustering Masks.

Zhaoheng Ni, Félix Grèzes, Viet Anh Trinh, Michael I. Mandel

CoRR, 2020

Enhancement of Spatial Clustering-Based Time-Frequency Masks using LSTM Neural Networks.

Félix Grèzes, Zhaoheng Ni, Viet Anh Trinh, Michael I. Mandel

CoRR, 2020

Mask-Dependent Phase Estimation for Monaural Speaker Separation.

Zhaoheng Ni, Michael I. Mandel

Proceedings of the 2020 IEEE International Conference on Acoustics, 2020

2019

Onssen: an open-source speech separation and enhancement library.

Zhaoheng Ni, Michael I. Mandel

CoRR, 2019

2018

Sound Signal Processing with Seq2Tree Network.

Weicheng Ma, Kai Cao, Zhaoheng Ni, Peter Chin, Xiang Li

Proceedings of the Eleventh International Conference on Language Resources and Evaluation, 2018

Unusable Spoken Response Detection with BLSTM Neural Networks.

Zhaoheng Ni, Rutuja Ubale, Yao Qian, Michael I. Mandel, Su-Youn Yoon, Abhinav Misra, David Suendermann-Oeft

Proceedings of the 11th International Symposium on Chinese Spoken Language Processing, 2018

2017

A Sep2Tree Model for Recognizing Synthetic Bach Chorales.

Xiuyan Ni, Weicheng Ma, Zhaoheng Ni, Robert M. Haralick

Proceedings of the 2017 International Computer Music Conference, 2017

Confused or not Confused?: Disentangling Brain Activity from EEG Data Using Bidirectional LSTM Recurrent Neural Networks.

Zhaoheng Ni, Ahmet Cem Yuksel, Xiuyan Ni, Michael I. Mandel, Lei Xie

Proceedings of the 8th ACM International Conference on Bioinformatics, 2017