Puyuan Peng

According to our database, Puyuan Peng authored at least 20 papers between 2020 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of four.

2024

SyllableLM: Learning Coarse Semantic Units for Speech Language Models.

Alan Baade, Puyuan Peng, David Harwath

CoRR, 2024

VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild.

Puyuan Peng, Po-Yao Huang, Daniel Li, Abdelrahman Mohamed, David Harwath

CoRR, 2024

BAT: Learning to Reason about Spatial Sounds with Large Language Models.

Zhisheng Zheng, Puyuan Peng, Ziyang Ma, Xie Chen, Eunsol Choi, David Harwath

Proceedings of the Forty-first International Conference on Machine Learning, 2024

SpeechCLIP+: Self-Supervised Multi-Task Representation Learning for Speech Via Clip and Speech-Image Data.

Hsuan-Fu Wang, Yi-Jen Shih, Heng-Jui Chang, Layne Berry, Puyuan Peng, Hung-Yi Lee, Hsin-Min Wang, David Harwath

Proceedings of the IEEE International Conference on Acoustics, 2024

AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models.

Yuan Tseng, Layne Berry, Yiting Chen, I-Hsiang Chiu, Hsuan-Hao Lin, Max Liu, Puyuan Peng, Yi-Jen Shih, Hung-Yu Wang, Haibin Wu, Poyao Huang, Chun-Mao Lai, Shang-Wen Li, David Harwath, Yu Tsao, Abdelrahman Mohamed, Chi-Luen Feng, Hung-Yi Lee

Proceedings of the IEEE International Conference on Acoustics, 2024

Integrating Self-Supervised Speech Model with Pseudo Word-Level Targets from Visually-Grounded Speech Model.

Hung-Chieh Fang, Nai-Xuan Ye, Yi-Jen Shih, Puyuan Peng, Hsuan-Fu Wang, Layne Berry, Hung-Yi Lee, David Harwath

Proceedings of the IEEE International Conference on Acoustics, 2024

Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos.

Changan Chen, Puyuan Peng, Ami Baid, Zihui Xue, Wei-Ning Hsu, David Harwath, Kristen Grauman

Proceedings of the Computer Vision - ECCV 2024, 2024

VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild.

Puyuan Peng, Po-Yao Huang, Shang-Wen Li, Abdelrahman Mohamed, David Harwath

Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024

2023

AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models.

Yuan Tseng, Layne Berry, Yi-Ting Chen, I-Hsiang Chiu, Hsuan-Hao Lin, Max Liu, Puyuan Peng, Yi-Jen Shih, Hung-Yu Wang, Haibin Wu, Po-Yao Huang, Chun-Mao Lai, Shang-Wen Li, David Harwath, Yu Tsao, Shinji Watanabe, Abdelrahman Mohamed, Chi-Luen Feng, Hung-yi Lee

CoRR, 2023

Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Mode.

Puyuan Peng, Shang-Wen Li, Okko Räsänen, Abdelrahman Mohamed, David Harwath

CoRR, 2023

Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalization.

Puyuan Peng, Brian Yan, Shinji Watanabe, David Harwath

Proceedings of the 24th Annual Conference of the International Speech Communication Association, 2023

Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model.

Puyuan Peng, Shang-Wen Li, Okko Räsänen, Abdelrahman Mohamed, David Harwath

Proceedings of the 24th Annual Conference of the International Speech Communication Association, 2023

Style-transfer based Speech and Audio-visual Scene understanding for Robot Action Sequence Acquisition from Videos.

Chiori Hori, Puyuan Peng, David Harwath, Xinyu Liu, Kei Ota, Siddarth Jain, Radu Corcodel, Devesh K. Jha, Diego Romeres, Jonathan Le Roux

Proceedings of the 24th Annual Conference of the International Speech Communication Association, 2023

Audio-Visual Neural Syntax Acquisition.

Cheng-I Jeff Lai, Freda Shi, Puyuan Peng, Yoon Kim, Kevin Gimpel, Shiyu Chang, Yung-Sung Chuang, Saurabhchand Bhati, David D. Cox, David Harwath, Yang Zhang, Karen Livescu, James R. Glass

Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop, 2023

2022

Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling.

Puyuan Peng, David Harwath

CoRR, 2022

Zero-shot Video Moment Retrieval With Off-the-Shelf Models.

Anuj Diwan, Puyuan Peng, Raymond J. Mooney

Proceedings of the Transfer Learning for Natural Language Processing Workshop, 2022

Word Discovery in Visually Grounded, Self-Supervised Speech Models.

Puyuan Peng, David Harwath

Proceedings of the 23rd Annual Conference of the International Speech Communication Association, 2022

MAE-AST: Masked Autoencoding Audio Spectrogram Transformer.

Alan Baade, Puyuan Peng, David Harwath

Proceedings of the 23rd Annual Conference of the International Speech Communication Association, 2022

Fast-Slow Transformer for Visually Grounding Speech.

Puyuan Peng, David Harwath

Proceedings of the IEEE International Conference on Acoustics, 2022

2020

A Correspondence Variational Autoencoder for Unsupervised Acoustic Word Embeddings.

Puyuan Peng, Herman Kamper, Karen Livescu

CoRR, 2020