Zhengyuan Yang

According to our database, Zhengyuan Yang authored at least 82 papers between 2015 and 2024.

Collaborative distances:

• Dijkstra number of four.
• Erdős number of three.

2024

Introduction to the Special Issue on AI-Generated Content for Multimedia.

Shengxi Li, Xuelong Li, Leonardo Chiariglione, Jiebo Luo, Wenwu Wang, Zhengyuan Yang, Danilo P. Mandic, Hamido Fujita

IEEE Trans. Circuits Syst. Video Technol., August, 2024

Multimodal Foundation Models: From Specialists to General-Purpose Assistants.

Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao

Found. Trends Comput. Graph. Vis., 2024

SlowFast-VGen: Slow-Fast Learning for Action-Driven Long Video Generation.

Yining Hong, Beide Liu, Maxine Wu, Yuanhao Zhai, Kai-Wei Chang, Lingjie Li, Kevin Lin, Chung-Ching Lin, Jianfeng Wang, Zhengyuan Yang, Yingnian Wu, Lijuan Wang

CoRR, 2024

EditRoom: LLM-parameterized Graph Diffusion for Composable 3D Room Layout Editing.

Kaizhi Zheng, Xiaotong Chen, Xuehai He, Jing Gu, Linjie Li, Zhengyuan Yang, Kevin Lin, Jianfeng Wang, Lijuan Wang, Xin Eric Wang

CoRR, 2024

MMCOMPOSITION: Revisiting the Compositionality of Pre-trained Vision-Language Models.

Hang Hua, Yunlong Tang, Ziyun Zeng, Liangliang Cao, Zhengyuan Yang, Hangfeng He, Chenliang Xu, Jiebo Luo

CoRR, 2024

Tuning Timestep-Distilled Diffusion Model Using Pairwise Sample Optimization.

Zichen Miao, Zhengyuan Yang, Kevin Lin, Ze Wang, Zicheng Liu, Lijuan Wang, Qiang Qiu

CoRR, 2024

AutoDirector: Online Auto-scheduling Agents for Multi-sensory Composition.

Minheng Ni, Chenfei Wu, Huaying Yuan, Zhengyuan Yang, Ming Gong, Lijuan Wang, Zicheng Liu, Wangmeng Zuo, Nan Duan

CoRR, 2024

MM-Vet v2: A Challenging Benchmark to Evaluate Large Multimodal Models for Integrated Capabilities.

Weihao Yu, Zhengyuan Yang, Linfeng Ren, Linjie Li, Jianfeng Wang, Kevin Lin, Chung-Ching Lin, Zicheng Liu, Lijuan Wang, Xinchao Wang

CoRR, 2024

VideoGUI: A Benchmark for GUI Automation from Instructional Videos.

Kevin Qinghong Lin, Linjie Li, Difei Gao, Qinchen Wu, Mingyi Yan, Zhengyuan Yang, Lijuan Wang, Mike Zheng Shou

CoRR, 2024

MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos.

Xuehai He, Weixi Feng, Kaizhi Zheng, Yujie Lu, Wanrong Zhu, Jiachen Li, Yue Fan, Jianfeng Wang, Linjie Li, Zhengyuan Yang, Kevin Lin, William Yang Wang, Lijuan Wang, Xin Eric Wang

CoRR, 2024

Motion Consistency Model: Accelerating Video Diffusion with Disentangled Motion-Appearance Distillation.

Yuanhao Zhai, Kevin Lin, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Chung-Ching Lin, David S. Doermann, Junsong Yuan, Lijuan Wang

CoRR, 2024

List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs.

An Yan, Zhengyuan Yang, Junda Wu, Wanrong Zhu, Jianwei Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Julian J. McAuley, Jianfeng Gao, Lijuan Wang

CoRR, 2024

Entity6K: A Large Open-Domain Evaluation Dataset for Real-World Entity Recognition.

Jielin Qiu, William Han, Winfred Wang, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Christos Faloutsos, Lei Li, Lijuan Wang

CoRR, 2024

Design2Code: How Far Are We From Automating Front-End Engineering?

Chenglei Si, Yanzhe Zhang, Zhengyuan Yang, Ruibo Liu, Diyi Yang

CoRR, 2024

StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesis.

Zecheng Tang, Chenfei Wu, Zekai Zhang, Mingheng Ni, Shengming Yin, Yu Liu, Zhengyuan Yang, Lijuan Wang, Zicheng Liu, Juntao Li, Nan Duan

CoRR, 2024

COSMO: COntrastive Streamlined MultimOdal Model with Interleaved Pre-Training.

Alex Jinpeng Wang, Linjie Li, Kevin Qinghong Lin, Jianfeng Wang, Kevin Lin, Zhengyuan Yang, Lijuan Wang, Mike Zheng Shou

CoRR, 2024

OpenLEAF: A Novel Benchmark for Open-Domain Interleaved Image-Text Generation.

Jie An, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Lijuan Wang, Jiebo Luo

Proceedings of the 32nd ACM International Conference on Multimedia, MM 2024, Melbourne, VIC, Australia, 28 October 2024, 2024

Bring Metric Functions into Diffusion Models.

Jie An, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Zicheng Liu, Lijuan Wang, Jiebo Luo

Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, 2024

MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities.

Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, Lijuan Wang

Proceedings of the Forty-first International Conference on Machine Learning, 2024

StrokeNUWA - Tokenizing Strokes for Vector Graphic Synthesis.

Zecheng Tang, Chenfei Wu, Zekai Zhang, Minheng Ni, Shengming Yin, Yu Liu, Zhengyuan Yang, Lijuan Wang, Zicheng Liu, Juntao Li, Nan Duan

Proceedings of the Forty-first International Conference on Machine Learning, 2024

IDOL: Unified Dual-Modal Latent Diffusion for Human-Centric Joint Video-Depth Generation.

Yuanhao Zhai, Kevin Lin, Linjie Li, Chung-Ching Lin, Jianfeng Wang, Zhengyuan Yang, David S. Doermann, Junsong Yuan, Zicheng Liu, Lijuan Wang

Proceedings of the Computer Vision - ECCV 2024, 2024

Idea2Img: Iterative Self-refinement with GPT-4V for Automatic Image Design and Generation.

Zhengyuan Yang, Jianfeng Wang, Linjie Li, Kevin Lin, Chung-Ching Lin, Zicheng Liu, Lijuan Wang

Proceedings of the Computer Vision - ECCV 2024, 2024

GRiT: A Generative Region-to-Text Transformer for Object Understanding.

Jialian Wu, Jianfeng Wang, Zhengyuan Yang, Zhe Gan, Zicheng Liu, Junsong Yuan, Lijuan Wang

Proceedings of the Computer Vision - ECCV 2024, 2024

MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning.

Chaoyi Zhang, Kevin Lin, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Chung-Ching Lin, Zicheng Liu, Lijuan Wang

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

Disco: Disentangled Control for Realistic Human Dance Generation.

Tan Wang, Linjie Li, Kevin Lin, Yuanhao Zhai, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, Lijuan Wang

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos.

Jielin Qiu, Jiacheng Zhu, William Han, Aditesh Kumar, Karthik Mittal, Claire Jin, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Ding Zhao, Bo Li, Lijuan Wang

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

Training Diffusion Models Towards Diverse Image Generation with Reinforcement Learning.

Zichen Miao, Jiang Wang, Ze Wang, Zhengyuan Yang, Lijuan Wang, Qiang Qiu, Zicheng Liu

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

Diagnostic Benchmark and Iterative Inpainting for Layout-Guided Image Generation.

Jaemin Cho, Linjie Li, Zhengyuan Yang, Zhe Gan, Lijuan Wang, Mohit Bansal

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

SGFormer: Semantic Graph Transformer for Point Cloud-Based 3D Scene Graph Generation.

Changsheng Lv, Mengshi Qi, Xia Li, Zhengyuan Yang, Huadong Ma

Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence, 2024

2023

TransVG++: End-to-End Visual Grounding With Language Conditioned Vision Transformer.

Jiajun Deng, Zhengyuan Yang, Daqing Liu, Tianlang Chen, Wengang Zhou, Yanyong Zhang, Houqiang Li, Wanli Ouyang

IEEE Trans. Pattern Anal. Mach. Intell., November, 2023

InfoVisDial: An Informative Visual Dialogue Dataset by Bridging Large Multimodal and Language Models.

Bingbing Wen, Zhengyuan Yang, Jianfeng Wang, Zhe Gan, Bill Howe, Lijuan Wang

CoRR, 2023

Interfacing Foundation Models' Embeddings.

Xueyan Zou, Linjie Li, Jianfeng Wang, Jianwei Yang, Mingyu Ding, Zhengyuan Yang, Feng Li, Hao Zhang, Shilong Liu, Arul Aravinthan, Yong Jae Lee, Lijuan Wang

CoRR, 2023

GPT-4V in Wonderland: Large Multimodal Models for Zero-Shot Smartphone GUI Navigation.

An Yan, Zhengyuan Yang, Wanrong Zhu, Kevin Lin, Linjie Li, Jianfeng Wang, Jianwei Yang, Yiwu Zhong, Julian J. McAuley, Jianfeng Gao, Zicheng Liu, Lijuan Wang

CoRR, 2023

GPT-4V(ision) as A Social Media Analysis Engine.

Hanjia Lyu, Jinfa Huang, Daoan Zhang, Yongsheng Yu, Xinyi Mou, Jinsheng Pan, Zhengyuan Yang, Zhongyu Wei, Jiebo Luo

CoRR, 2023

MM-VID: Advancing Video Understanding with GPT-4V(ision).

Kevin Lin, Faisal Ahmed, Linjie Li, Chung-Ching Lin, Ehsan Azarnasab, Zhengyuan Yang, Jianfeng Wang, Lin Liang, Zicheng Liu, Yumao Lu, Ce Liu, Lijuan Wang

CoRR, 2023

DEsignBench: Exploring and Benchmarking DALL-E 3 for Imagining Visual Design.

Kevin Lin, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Lijuan Wang

CoRR, 2023

Idea2Img: Iterative Self-Refinement with GPT-4V(ision) for Automatic Image Design and Generation.

Zhengyuan Yang, Jianfeng Wang, Linjie Li, Kevin Lin, Chung-Ching Lin, Zicheng Liu, Lijuan Wang

CoRR, 2023

OpenLEAF: Open-Domain Interleaved Image-Text Generation and Evaluation.

Jie An, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Lijuan Wang, Jiebo Luo

CoRR, 2023

The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision).

Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, Lijuan Wang

CoRR, 2023

Spatial-Frequency U-Net for Denoising Diffusion Probabilistic Models.

Xin Yuan, Linjie Li, Jianfeng Wang, Zhengyuan Yang, Kevin Lin, Zicheng Liu, Lijuan Wang

CoRR, 2023

DisCo: Disentangled Control for Referring Human Dance Generation in Real World.

Tan Wang, Linjie Li, Kevin Lin, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, Lijuan Wang

CoRR, 2023

MultiSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos.

Jielin Qiu, Jiacheng Zhu, William Han, Aditesh Kumar, Karthik Mittal, Claire Jin, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Bo Li, Ding Zhao, Lijuan Wang

CoRR, 2023

NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation.

Shengming Yin, Chenfei Wu, Huan Yang, Jianfeng Wang, Xiaodong Wang, Minheng Ni, Zhengyuan Yang, Linjie Li, Shuguang Liu, Fan Yang, Jianlong Fu, Gong Ming, Lijuan Wang, Zicheng Liu, Houqiang Li, Nan Duan

CoRR, 2023

MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action.

Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, Lijuan Wang

CoRR, 2023

Revisiting Transformer for Point Cloud-based 3D Scene Graph Generation.

Changsheng Lv, Mengshi Qi, Xia Li, Zhengyuan Yang, Huadong Ma

CoRR, 2023

Learning 3D Photography Videos via Self-supervised Diffusion on Single Images.

Xiaodong Wang, Chenfei Wu, Shengming Yin, Minheng Ni, Jianfeng Wang, Linjie Li, Zhengyuan Yang, Fan Yang, Lijuan Wang, Zicheng Liu, Yuejian Fang, Nan Duan

Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, 2023

Prompting GPT-3 To Be Reliable.

Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan L. Boyd-Graber, Lijuan Wang

Proceedings of the Eleventh International Conference on Learning Representations, 2023

Equivariant Similarity for Vision-Language Foundation Models.

Tan Wang, Kevin Lin, Linjie Li, Chung-Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, Lijuan Wang

Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023

PromptCap: Prompt-Guided Image Captioning for VQA with GPT-3.

Yushi Hu, Hang Hua, Zhengyuan Yang, Weijia Shi, Noah A. Smith, Jiebo Luo

Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023

ReCo: Region-Controlled Text-to-Image Generation.

Zhengyuan Yang, Jianfeng Wang, Zhe Gan, Linjie Li, Kevin Lin, Chenfei Wu, Nan Duan, Zicheng Liu, Ce Liu, Michael Zeng, Lijuan Wang

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation.

Shengming Yin, Chenfei Wu, Huan Yang, Jianfeng Wang, Xiaodong Wang, Minheng Ni, Zhengyuan Yang, Linjie Li, Shuguang Liu, Fan Yang, Jianlong Fu, Ming Gong, Lijuan Wang, Zicheng Liu, Houqiang Li, Nan Duan

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023

2022

GIT: A Generative Image-to-text Transformer for Vision and Language.

Jianfeng Wang, Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, Lijuan Wang

Trans. Mach. Learn. Res., 2022

PromptCap: Prompt-Guided Task-Aware Image Captioning.

Yushi Hu, Hang Hua, Zhengyuan Yang, Weijia Shi, Noah A. Smith, Jiebo Luo

CoRR, 2022

Cross-modal Contrastive Distillation for Instructional Activity Anticipation.

Zhengyuan Yang, Jingen Liu, Jing Huang, Xiaodong He, Tao Mei, Chenliang Xu, Jiebo Luo

Proceedings of the 26th International Conference on Pattern Recognition, 2022

Apple Counting Network Before Fruit Thinning Period Based On Dilated Convolution.

Mengyang Song, Guoquan Jiang, Zhanqiang Huo, Zhengyuan Yang, Hongxu Zhang

Proceedings of the 11th International Conference on Networks, Communication and Computing, 2022

UniTAB: Unifying Text and Box Outputs for Grounded Vision-Language Modeling.

Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Faisal Ahmed, Zicheng Liu, Yumao Lu, Lijuan Wang

Proceedings of the Computer Vision - ECCV 2022, 2022

Scaling Up Vision-Language Pretraining for Image Captioning.

Xiaowei Hu, Zhe Gan, Jianfeng Wang, Zhengyuan Yang, Zicheng Liu, Yumao Lu, Lijuan Wang

Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA.

Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Yumao Lu, Zicheng Liu, Lijuan Wang

Proceedings of the Thirty-Sixth AAAI Conference on Artificial Intelligence, 2022

2021

Grounding-Tracking-Integration.

Zhengyuan Yang, Tushar Kumar, Tianlang Chen, Jingsong Su, Jiebo Luo

IEEE Trans. Circuits Syst. Video Technol., 2021

Scaling Up Vision-Language Pre-training for Image Captioning.

Xiaowei Hu, Zhe Gan, Jianfeng Wang, Zhengyuan Yang, Zicheng Liu, Yumao Lu, Lijuan Wang

CoRR, 2021

Crossing the Format Boundary of Text and Boxes: Towards Unified Vision-Language Modeling.

Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Faisal Ahmed, Zicheng Liu, Yumao Lu, Lijuan Wang

CoRR, 2021

UFO: A UniFied TransfOrmer for Vision-Language Representation Learning.

Jianfeng Wang, Xiaowei Hu, Zhe Gan, Zhengyuan Yang, Xiyang Dai, Zicheng Liu, Yumao Lu, Lijuan Wang

CoRR, 2021

SAT: 2D Semantics Assisted Training for 3D Visual Grounding.

Zhengyuan Yang, Songyang Zhang, Liwei Wang, Jiebo Luo

Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, 2021

TransVG: End-to-End Visual Grounding with Transformers.

Jiajun Deng, Zhengyuan Yang, Tianlang Chen, Wengang Zhou, Houqiang Li

Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, 2021

TAP: Text-Aware Pre-Training for Text-VQA and Text-Caption.

Zhengyuan Yang, Yijuan Lu, Jianfeng Wang, Xi Yin, Dinei Florêncio, Lijuan Wang, Cha Zhang, Lei Zhang, Jiebo Luo

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021

Improving Weakly Supervised Visual Grounding by Contrastive Knowledge Distillation.

Liwei Wang, Jing Huang, Yin Li, Kun Xu, Zhengyuan Yang, Dong Yu

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2021

2020

Dynamic Context-guided Capsule Network for Multimodal Machine Translation.

Huan Lin, Fandong Meng, Jinsong Su, Yongjing Yin, Zhengyuan Yang, Yubin Ge, Jie Zhou, Jiebo Luo

Proceedings of the MM '20: The 28th ACM International Conference on Multimedia, 2020

Weakly Supervised Body Part Segmentation with Pose based Part Priors.

Zhengyuan Yang, Yuncheng Li, Linjie Yang, Ning Zhang, Jiebo Luo

Proceedings of the 25th International Conference on Pattern Recognition, 2020

Pose-based Body Language Recognition for Emotion and Psychiatric Symptom Interpretation.

Zhengyuan Yang, Amanda Kay, Yuncheng Li, Wendi Cross, Jiebo Luo

Proceedings of the 25th International Conference on Pattern Recognition, 2020

Improving One-Stage Visual Grounding by Recursive Sub-query Construction.

Zhengyuan Yang, Tianlang Chen, Liwei Wang, Jiebo Luo

Proceedings of the Computer Vision - ECCV 2020, 2020

A Novel Graph-based Multi-modal Fusion Encoder for Neural Machine Translation.

Yongjing Yin, Fandong Meng, Jinsong Su, Chulun Zhou, Zhengyuan Yang, Jie Zhou, Jiebo Luo

Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020

2019

Action Recognition With Spatio-Temporal Visual Attention on Skeleton Image Sequences.

Zhengyuan Yang, Yuncheng Li, Jianchao Yang, Jiebo Luo

IEEE Trans. Circuits Syst. Video Technol., 2019

Grounding-Tracking-Integration.

Zhengyuan Yang, Tushar Kumar, Tianlang Chen, Jiebo Luo

CoRR, 2019

Weakly Supervised Body Part Parsing with Pose based Part Priors.

Zhengyuan Yang, Yuncheng Li, Linjie Yang, Ning Zhang, Jiebo Luo

CoRR, 2019

Human-Centered Emotion Recognition in Animated GIFs.

Zhengyuan Yang, Yixuan Zhang, Jiebo Luo

Proceedings of the IEEE International Conference on Multimedia and Expo, 2019

A Fast and Accurate One-Stage Approach to Visual Grounding.

Zhengyuan Yang, Boqing Gong, Liwei Wang, Wenbing Huang, Dong Yu, Jiebo Luo

Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, 2019

Attentive Relational Networks for Mapping Images to Scene Graphs.

Mengshi Qi, Weijian Li, Zhengyuan Yang, Yunhong Wang, Jiebo Luo

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019

2018

End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perception.

Zhengyuan Yang, Yixuan Zhang, Jerry Yu, Junjie Cai, Jiebo Luo

CoRR, 2018

End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perceptions.

Zhengyuan Yang, Yixuan Zhang, Jerry Yu, Junjie Cai, Jiebo Luo

Proceedings of the 24th International Conference on Pattern Recognition, 2018

Action Recognition with Visual Attention on Skeleton Images.

Zhengyuan Yang, Yuncheng Li, Jianchao Yang, Jiebo Luo

Proceedings of the 24th International Conference on Pattern Recognition, 2018

2017

Personalized pose estimation for body language understanding.

Zhengyuan Yang, Jiebo Luo

Proceedings of the 2017 IEEE International Conference on Image Processing, 2017

2015

Curve fitting and optimal interpolation for CNC machining under confined error using quadratic B-splines.

Zhengyuan Yang, Liyong Shen, Chun-Ming Yuan, Xiao-Shan Gao

Comput. Aided Des., 2015